MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation
Authors: Hao Shi
Bin Xie
Yingfei Liu
Lin Sun
Fengrong Liu
Tiancai Wang
Erjin Zhou
Haoqiang Fan
Xiangyu Zhang
Gao Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Embodied AI
Vision-Language-Action Models
robotic manipulation
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose MemoryVLA, a Cognition-Memory-Action framework for robotic manipulation, inspired by human hippocampus mechanism.

Abstract: Temporal context is essential for robotic manipulation because such tasks are inherently non-Markovian, yet mainstream VLA models typically overlook it and struggle with long-horizon, temporally dependent tasks. Cognitive science suggests that humans rely on working memory to buffer short-lived representations for immediate control, while the hippocampal system preserves verbatim episodic details and semantic gist of past experience for long-term memory. Inspired by these mechanisms, we propose MemoryVLA, a Cognition-Memory-Action framework for long-horizon robotic manipulation. A pretrained VLM encodes the observation into perceptual and cognitive tokens that form working memory, while a Perceptual-Cognitive Memory Bank stores low-level details and high-level semantics consolidated from it. Working memory retrieves decision-relevant entries from the bank, adaptively fuses them with current tokens, and updates the bank by merging redundancies. Using these tokens, a memory-conditioned diffusion action expert yields temporally aware action sequences. We evaluate MemoryVLA on 150+ simulation and real-world tasks across three robots. On SimplerEnv-Bridge, Fractal, LIBERO-5 suites and Mikasa-Robo, it achieves 71.9%, 72.7%, 96.5%, and 41.2% success rates, respectively, all outperforming state-of-the-art baselines CogACT and pi-0, with a notable +14.6 gain on Bridge and +11.8 gain on Mikasa-Robo. On 12 real-world tasks spanning general skills and long-horizon temporal dependencies, MemoryVLA achieves 84.0% success rate, with long-horizon tasks showing a +26 improvement over state-of-the-art baseline. Project Page: https://shihao1895.github.io/MemoryVLA

Published as a conference paper at ICLR 2026
VEM: PERCEPTUAL-COGNITIVE MEM- ORY IN VISION-LANGUAGE-ACTION MODELS FOR
ROBOTIC MANIPULATION
Hao Shilt it
Bin Xie
Yingfei Liu Lin Sun +t
Fengrong Liu Gao Huangle
Tiancai Wang?
Erjin Zhou² Haoqiang Fan Xiangyu Zhang
DDpartmeet of Automation, BNRist, Tsinghua University Dexmal
3 MEEVVI Technology TTiajin University Harbin Institute Technology SStepFun shi-h23 @ma 1s. testtiingg Esinguua. edu. gaohuange ts inghuaa edu.cn
Work done during interning at Dexmal.
e Corresponding author.
ABSTRACT
Temporal context is essential for robotic manipulation because such tasks are in- herently non-Markovian, yet mainstream VLA models typically overlook it and struggle with long-horizon, temporally dependent tasks. Cognitive science sug- gests that humans rely on working memory to buffer short-lived representations for immediate control, while the hippocampal system preserves verbatim episodic details and semantic gist of past experience for long-term memory. Inspired by these mechanisms, we propose MemorryVLA, a Dremition-Memory-AAtiio framework for long-horizon robotic manipulation. A pretrained VLM encodes the observation into perceptual and cognitive tokens that form working memory. while Parceptual-Cogniing Memory Bank stores low-level details and high- level semantics consolidated from Working memory retrieves decision-relevant entries from the bank, adaptively fuses them with current tokens, and updates the bank by merging redundancies. Using these tokens, a memory-conditioned diffusion action expert yields temporally aware action sequences. We evalu- ate MemoryVLA on 150+ simulation and real-world tasks across three robots. On SimplerEnv-Bridge Fractal, LIBERO-5 suites and Mikasa Roboo, it achieves 71.9%, 72.7%, 96.5%, and 41.2% success rates, respectively, all outperforming state-of-the-ard baselines CogACT and To, with notable +14.6 gain Bridge and +11.8 gain on Mikasa-Robo. On 12 real-world tasks spanning general skills and long-horizon temporal dependencies, Memory VLA achieves 84.0% success score, with long-horizon tasks showing a +26 improvement over state-of-the-arr baseline. Project Page: https://shihao 895, 19.. github.io//emorry VLA
INTRODUCTION
Vision-Language-Action (VLA) models (Brohan et al., 2023; Kim et al., 2024; Black et al., 2024; Li al., 2024a; Sun al., 2025; Xie et al., 2025), powered by large-scale cross-embodiment robotic datasets (O'Neill et al., 2024; Brohan et al., 2022; Khazatsky et al., 2024; Bu et al., 2025a) and pretrained Vision Languu- Models (VLMs) (Karamcheti et al., 2024: Liu et al., 2023b; Bai et al., 2023a), have achieved remarkable progress in robotic manipulation. However, mainstream VLA models such as VLA (Kim et al., 2024) and To (Black et al., 2024) rely solely on the current observation, thereby overlooking temporal dependencies and performing poorly on long-horizon temporal manipulation tasks. As shown in Fig. (a), Push Buttons tasks exhibit almost no visual difference before and after pushing, making it difficult to determine whether the action has already been completed. This highlights the non-M Non-Makoovinn nature of manipulation, where earlier actions influence later decisions, calling for temporal modeling. A naive strategy to concatenate con- secutive frames as input to the VLM. However, it faces two critical limitations: (1) The quadratic complexity of self-attention severely limits the usable temporal context length; (2) Sequential frame inputs are misaligned with the model's single-frame robotic pretraining distribution.

---

Published as a conference paper at ICLR 2026
(4) Example of Temporal Confusion
Ready 6OI
Move Down
Move Up
Push Repeatedly
Hove I already pushed it?
(b) Human wwwiiiiioo-Mmmor--oo
(c) MemoryVLA
Neural Activity
Working Memory
Action
peroeption
Workingg Menoryy
Persspiual
Expert
Cerebellum
Cogntivv Takes
Retrinal
Action
Text
Vision Encoder
Fusion
Update
Hippocampus (Episodic Memoryy
Prompt
Porceptial- Cognltive Memory Bank
Verbarim Representation
LoweLosst
Representation
LLM
Cognition
Memory
High-l Semantics
(d) Strang Performance
BastaMLA
42.7
Gperit
75.0 TAFAST
SprravtA
21.0
Operita
21
OpenMLA 9
CogACT
57
83.1
33
CogACT
28.4
72
$2
68.4
CogACT
93.2
29.4
ERgACT
76
CogAecT
57
71.9
41.2
70 No 90 (00 LIBERO
45
30
SimplerEnv
Mikasa-Rod Robo
Real-worll--ennnal
Real-world- Reul-wworid--rrrppnn
Figure (a) In Push Buttons tasks, pre- and post-push states look nearly identical, calling for temporal modeling. (b) Humans handle manipulation tasks via a dual-memory system: working memory (neural activity) supports short-term control, while episodic memory (hippocampus) pre- serves long-term experience.
(c) Inspired by this, MemoryVLA introduces a Perceptual-Cognitte Memory Bank that consolidates low-level perceptual details and high-level cognitive semantics for temporally aware decision making. (d) Memory VLA outperforms state-of-ther thee--r-the-rtt baselines.
Research in cognitive science (Baddeley & Hitch, 1974; Tulving et al., 1972; Reyna Brain- erd, 1995) demonstrates that humans handle manipulation tasks through dual-memory system (Fig. (b)). The brain encodes multi-modal sensory inputs into both perceptual and cognitive rep- resentations. These representations are buffered in working memory via transient neural activity, providing short-term retention for immediate decision-makion makingg Concurrently, episodic memory, the long-term memory system supported by hippocampus, encodes past experiences with temporal index in two forms: verbatim representations preserving precise details and gist representations cap- turing abstract semantics. During execution, working memory retrieves decision-relevant contexts from episodic memory and integrates them with current representations guide actions through cerebellar control, while simultancously consolidating new experiences into episodic memory.
Drawing on cognitive science insights, we propose MemoryVLA (Fig. (c)), Cognition-Memory Action framework for robotic manipulation that explicitly models temporal dependencies through Perceptual Cronntual-Cooniitive Memory Bank (PCMB). First, vision encoder extracts perceptual tokens from observation, while a large language model (LLM) processes them together with the language instruction, leveraging commonsense priors produce cognitive tokens. Perceptual and cognitive tokens jointly form the working memory. Second, the PCMB stores both low-level perceptual details and high-level cognitive semantics over long horizons. During retrieval, working memory buffers current tokens and queries the PCMB with temporal positional encodings to fetch decision-relevant historical contexts, which are adaptively fused with current tokens via gating mechanism while si- multaneously updating the PCMB. When capacity is reached, temporally adjacent and semantically similar entries are consolidated to preserve essential information compactly. Finally, memory- conditioned diffusion action expert is conditioned on cognitive tokens, with perceptual tokens en- riching them with fne-graaiee details, to produce temporally aware robotie action sequences.
We conduct extensive evaluations of Memory VLA across 3 robots and 150+ tasks with 500+ varia- tions in simulation and real world. For SimplerEnv (Li al., 2024b), Memory VLA achieves 71.9% and 72.7 727% success rates Bridge and Fractal suites, surpassing CogACT by 14.6 and 4.6 points, further outperforming Ty. For LIBERO (Liu et al., 2023a), it achieves 96.5% success rate across
2

---

Published as a conference paper at ICLR 2026
suites (Spatial, Object, Goal, Long-10, and Long-90), exceeding both CogACT and To. For Mikasa- Robo (Cherepanov et al., 2025). achieves 41.2% success rate, outperforming To by 11.8 points. For real-world evaluations, we introduce 12 tasks actoss Franka and WidowX robots, spanning 6 general tasks and 6 long-horizon temporal tasks. MemoryVI.A achieves 85% and 83% scores on general and temporal tasks, outperforming CogACTI by 9 and 26 points, and substantially surpassing Moreover, Memory VLA exhibits strong robustness and generalization under out-of-distributis conditions involving varied backgrounds, distractors, objects, containers, lighting and occlusion.
Our contributions are summarized as follows:
Inspired by human memory systems from cognitive science, we propose NemoryVI.A, Cognitinn-eemory-Ation framework that leverages VLM commonsense priors, a perceptual- cognitive memory mechanism, and diffusion action expert to capture long-horizon temporal dependencies for robotic manipulation.
We design a Perceptual- Cognitive Memory Bank with working memory that enables memory re- trieval of decision-relevant contexts across high-level cognition and low-level perception, memory fusion that adaptively integrates them with current representations, and memory consolidation that merges temporally adjacent, semantically similar entries.
Memory VLA achieves state-of-the-an performance on SimplerEnv, LIBERO, Mikasa-Robo and real-world. It also demonstrates strong robustness and generalization. On challenging long- horizon real-world tasks, it outperforms CogACT and Ti by significant margins, underscoring the importance of temporal memory modeling.
2 RELATED WORKS
Lanoon-aang Vision-Language-Action Models Driven by advances in visual foundation models (Radford et al., 2021; Caron et al., 2021; Liu et al., 2024a; Zheng et al., 2024a; 2025a; Zhang et al., 2025b; Wu et al.,
2025; Wang al., 2025), robot imitation learning has progressed rapidly yet remains confined to small, task-specific policies with limited generalization (Shridhar et al., 2023; Zhao et al., 2023; Chi al., 2023; Goyal et al., 2023; Shi al., 2025). To overcome these, the success of VLMs (Achiam et al., 2023; Touvron et al., 2023; Liu et al., 2023b; Bai et al., 2023b; Guo et al., 2025) and large- scale robot datasets (e-g.. OXE (O'Neill et al., 2024), Agibot (Bu al., 2025a)) spawned the vision- language-action (VLA) paradigm (Kim et al., 2024; Black et al., 2024; Yue et al., 2024; Zhong et al., 2025; Gao et al., 2025; Yang et al., 2025b). RT-2 (Zitkovich et al., 2023) and Open (Kim et al., 2024) tokenize continuous actions into discrete tokens and use VLMs for autoregressive pre- diction as if generating language. In contrast, Tu (Black et al., 2024), CogACT (Li et al., 2024a), Dex (Wen et al., 2025) and HybridVLA (Liu et al., 2025c) adopt diffusion- based policies (Chi et al., 2023; Liu et al., 2024b) as action heads, leveraging iterative denoising to sample continuous control trajectories that capture diverse multimodal behaviors. However, none of these methods explicitly model temporal dependencies. Robotic manipulation is inherently non-Markovian, and neglecting history leads to failures on long-horizon temporal tasks.
Temporal Modeling in Robotics Temporal modeling has been extensively studied in computer vision and autonomous driving (Wang et al., 2023: Liu et al., 2023c; Feng et al., 2023; Zhou et al., 2024), yet it has not been fully explored in robotic manipulation. Octo (Mees et al., 2024), RoboVLMs (Liu et al., 2025b), and Interleave-VL. VLA (Fan et al., 2025) adapt the VLM paradigm to model robotic video data in interleaved image-text format. While conceptually elegant, this format is complex to implement and computationally expensive, hindering its widespread applica- tion. RoboFlamingo (Li et al., 2023) compresses vision-language representation into latent token and propagate it via LSTM (Hochreiter & Schmidhuber, 1997). The latent representation is ob- tained in a relatively coarse manner and the fine-grained perceptual history is largely discarded. TraceVLA (Zheng et al., 2024b) takes a different route, painting historical states as trajectories on the current frame, yet discards rich semantic details. UniVLA (Bu et al., 2025b) incorporates past ac- tions into input prompts, making an initial attempt at temporal modeling. However, it merely serves as Shain-of-Thoughe (Wei et al., 2022) process without effectively utilizing historical information. In contrast, we model both high-level cognitive semantics and fine- ine-ggained perceptual details within a memory framework, enabling effective temporal modeling for long-horizon manipulation.
3

---

Published as a conference paper at ICLR 2026
Perceptual token Cognitive token
Pereeptual-Cognitive
3
Memory Bank
-/
Cognition Attention
Lowe-Level Decails
Clean the table und push button for count.
Mcmory
Add & Norm
Vision Encoder
Consolidation
High-L Semantics
Perception Allention
Projector
Text Tokenizer
Add & Norm
Timestop PE
Large Language Model
Memory Retrieval
Mernoyy
Add & Norm
Perceptual
Working Memory
Gute Fusion
Compression
Query
MLP
Figure 2: Overall architecture MemoryVLA, RGB observation and language instruction are encoded by 7B VLM into perceptual and cognitive tokens, forming short-term working memory. The working memory queries a perceptual-cognniive memory bank (PCMB) to retrieve relevant his- torical context, including high-level semantics and low-level visual details, adaptively fuses it with current tokens, and consolidates the PCMB by merging the most similar neighbors. The memory- augmented tokens then condition diffusion transformer predict a sequence of future actions.
METHOD
3.1
OVERVIEW OF MEMORYVLA
Problem Formulation We formulate robotic manipulation in VLA models as sequential decision-making process, where visual observations and language instructions are mapped to con- trol actions for real world interaction. Given the current RGB image IR x W and language instruction L, parameterized policy æ outputs sequence of future actions
= (01, (01,..., ,T) = [),
where each action a; [Ax, Ay, Az, A0., A0,, A0,,g] consists of relative translation, relative rotation (Euler angles), and a binary gripper state E (0, 1}.
Overview Memory VI.A Á is end-to-end framework for robotic manipulation, shown in Fig. 2. The current RGB observation and language instruction are first encoded by VLM into percep- tual and cognitive tokens, forming a working memory, analogous neural activity in the visual and prefrontal cortex associated with short-term memory. To complement this short-term store, we introduce the Perceptual PereeptualCoonnttive Memory Bank (PCMB), inspired by the hippocampus, which maintains long-term high-level semantics and fine-grate fine-grained perceptual details. Working-memory embeddings query PCMB to retrieve decision-relevant history, adaptively fuse with current representations via gating, and consolidate the memory by merging temporally adjacent and seman- tically similar entries when capacity is reached. The resulting representations are then fed into a memory-conditionee diffusion action expert to generate sequence of N future 7-DoF actions.
3.2
VISION-LANGUACE COGNITION MODULE
We build upon a 7 7B-parameter Prismatic VLM (Karamcheti et al., 2024), which is further pretrained on the large-scale cross-embodiment real robot dataset Open-X Embodiment (O'Neill al., 2024). For visual encoding, we adopt parallel DINOv2 (Oquab et al., 2023) and SigLIP (Zhai et al., 2023) backbones on the current third-person RGB image T, concatenating their features into raw visual tokens. A perceptual compression module, implemented via SE-bottleneck (Hu et al., 2018), then compresses these tokens into a compact set of perceptual tokens p E Xd, with N, = 256. In parallel, the raw visual tokens are projected via linear layer into the language embedding space and concatenated with the tokenized instruction before being fed into the LLaMA-7B (Touvron et al., 2023). The output at the end-of-sentence (EOS) position is taken the cognitive token R xd,"" representing high-level cognitive semantics in compact form. Finally, the perceptual tokens p and cognitive token are combined to form the short-term working memory for downstream modules.
4

---

Published as a conference paper at ICLR 2026
Push
Cognitive Memory
Concat
Perceptual al Memory
Timestep
MLP
Yes
Full
Emb
Cross
Sigmoid
Attention
Positional
FFN
Embedding
Mergo
Argmmu
(a) Memory Retrieval
Memory Gate Fusion
Memory Consolidation
current perceptual and cognitive tokens query the PCMB via cross-attention with timestep positional encoding fetch relevant historical features. (b) Gate fusion: current and retrieved tokens are adaptively fused via gate mechanism. (c) Con- solidation: the fused tokens are updated into PCMB. When PCMB reaches its capacity, we compute similarities between adjacent entries and merge the most similar pair to maintain compactness.
Figure 3: Details of memory module. (a) Retrieval:
3.3 PERCEPTUAL-COGNIIIII MEMORY MODULE
The Vision-Language Cognition Module yields working memory
Mwk {p E Xd,,, e R'xd),,
(2)
where p and e represent the current perceptual tokens and cognition token, respectively. However, this working memory only reflects the present timestep and lacks temporal dependencies.
To address this, inspired by the hippocampus in human memory systems, we introduce the Percep- tual-Cognitive Memory Bank (PCMB):
Mpemb { mm {perr cog] },
(3)
m = {m, E N.N xd, Hi11
(pper,cog),
(4)
where each perceptual entry m. stores fine-grained visual details and each cognitive entry my en- codes high-level semantic summary. The bank maintains up to L. entries per stream.
Memory Retrieval At each timestep, the working memory Mwk. comprising current perceptual tokens E Ny and cognition token E IR 1xd. acts as a dual query to retrieve historical in- formation required for the current decision from the Perceptual- Coogntuul- Memory Bank Mpemb as illustrated in Fig. 3 (a). Each memory entry is associated with its episode timestep via sinusoidal embedding TE(-), which is added as positional encoding. We then stack all perceptual memories into tensor R NN, xd, and cognitive memories into tensor E RLdde Scaled dot-product atten- tion between the current tokens and these memory tensors produces raw outputs for both streams:
K
[m]] + TE(()); ...; mi + TE((L)). V y== [mm]: m 1
(5)
A softmax (K*)))
V2,
e tp. c). IE {per, cog}.
(6)
Vdn
This attention operation is followed by a feed-forward network to complete one Transformer layer, and applying two such layers yields the final retrieved embeddings HP and H
Memory Gate Fusion As illustrated in Fig. 3 (b), the gate fusion process integrates the retrieved embeddings Hº and He with the current working memory representations through learned gates. For both the perceptual (r p) and cognitive (r e) streams, gating vector is computed
g" (MLP(concat(z, H)))),
and applied obtain the memory cemmory-augmenterr representation
= o" c H (1 0z)
Here, a denotes the sigmoid activation and c denotes element wise multiplication. The resulting memory-augmented features ğ and ē are then forwarded the memory consolidation stage.
5

---

Published as a conference paper at ICLR 2026
Memory Consolidation After gate fusion, the memory mamnory-uugmetter representations P and č are passed to the Memory-conditione Action Expert and simultaneously updated to the PCMB. When the number of stored entries exceeds L, cosine similarities are computed within each stream (per- ceptual and cognitive) between adjacent entries. The pair with the highest similarity in each stream selected and merged by averaging their vectors, thereby reducing redundancy.
arg m cos 11+1), m + 2 (āx; + E,z+1), E {per, cog). (9)
This consolidation mechanism (Fig. 3 (c)) mitigates memory bloat by reducing redundancy, while preserving the most salient perceptual details and semantic abstractions, thereby maintaining com- pact representation that supports efficient long-term memory.
3.4
ACTION EXPERT
Leveraging the memory-augmenter working memory {p. ē). which integrates historical perceptual and cognitive information, the action expert predicts a sequence of future actions {ai, a2,.... aT), with T = 16. This prediction allows the model to anticipate multi-step trajectories, reduce cumu- lative error, and provide foresight for long-horizon execution. Since real-world robotic actions lie in continuous multimodal control space, We adopt diffusion-based Transformer (DiT) (Peebles & Xie, 2023) implemented with Denoising Diffusion Implicit Models (DDIM) (Song et al., 2020), using 10 denoising steps for efficient yet accurate trajectory generation. This architecture progres- sively denoises sequence of noisy action tokens, yielding precise continuous actions.
Specifically, at each denoising step, the noisy action tokens are injected with the sinusoidal encoding of the denoising timestep and concatenated with the cognitive representation č. A cognition-altention layer conditions the process with high-level semantic guidance, while perception attenntion layer supplements fine-gre inn-grrined visual details from the perceptual features P. The combined representation then refined through a ferd-forward network obtaín the denoised action at that step. The model is trained with mean squared error (MSE) loss between the predicted and target actions, and the final denoised vectors are passed through an MLP to generate continuous 7-DoF robotic actions.
EXPERIMENTS
To comprehensively evaluate Memory we organize experiments around six core questions: (1) How does MemoryVLA compare with state-of-the-art methods on SimplerEnv benchmark? (Sec. 4.2) (2) How does it perform on LIBERO benchmark? (Sec. 4.3) (3) How does it perform on Mikasa- Robo benchmark? (Sec. 4.4) (4) Can it handle both general manipulation and long- morizoo
temporal tasks on real robots? (Sec. 4.5) (5) What the impact of each component? (Sec. 4.6) (6) How robust and generalizable is under diverse environmental conditions? (Appendix B)
4.1 EXPERIMENTAL SETUPS
Simulation and Real-world Benchmarks. Fig. 4 overviews our evaluation across simulation and real-world, covering 3 robots, 6 benchmarks, 150+ tasks with 500+ variations. SimplerEnv (Li al., 2024b) includes Bridge suite with WidowX robot and Fractal suite with Google robot. Fractal provides two settings: Visual Matching (VM) and Visual Aggregation ( (VA). LIBERO (Liu et al., 2023a) uses a Franka robot and spans five suites (Spatial, Object, Goal, Long- 10, and Long-90). Mikasa-Robo (Cherepanov et al., 2025) uses a Franka robot. In real-world, we evaluate General and Long-horizon Temporal suites on Franka and WidowX robots. Task details for each benchmark and additional qualitative results are provided in Appendix and Appendix M.
Implementation Details We train on 8 NVIDIA A100 GPUs with PyTorch FSDP, using 32 sam- ples per GPU for global batch of 256 and learning rate of 2 x 10-5 The model takes a single third-person RGB frame at 224 x 224 together with the language instruction and outputs 7-DoF actions. The LLM is 7B, and the diffusion action expert has ~300M parameters. At inference we use DDIM (Song et al., 2020) with sampling steps and classifier-fres guidance(CFG) (Ho & Salimans, 2022) guidance scale of 1.5. Additional details are provided in Appendix C and D.

---

Published as a conference paper at ICLR 2026
Robots, Benchmarks, 150+ Tasks, 500+ Variations
SimplerEnv-Bridge
SimplerEnv-rractal
LIBERO
Mikasa-Robo
Spucn on Towall
Canot oz Plate
Coke Can
Move Near
Spanel Soite
Oboon Suite
Cheme
Intenopp
Long-100
Real-world Robots
Robustness Generalization
Insen Cincle
Eig
Stack Cups
Stuck Bincks
Pick Divene Fruis
Background
Dotracions
Lightng
Seg Post Buttons
Charge Food
Guess Where
Clean Table Count
Pack Place Order
Clean Rest Tike
Object
Container
Occlusion
Figure 4: Experimental setup overview. Top: Four simulation benchmarks, SimpleEnv-Ben -Bridg,, SimpleEnv-Frace Fractal, LIBERO, and Mikasa- Robo. Bottom: real-world evaluation (General and Long- horizon Temporal), real-world robustness and generalization evaluation. total, we evaluate robots across 6 benchmarks, spanning over 150 tasks and 500 variations.
4.2 SIMULATED EVALUATION ON SIMPLERENV
Training and Evaluation Setup CaaplerEnv-Bridgee we train Bridge v2 dataset (Walke et al., 2023) for 50k steps with validation every 2.5k steps. Results are reported at the best validation step, and each task evaluated with 24 trials to compute success rates. For SimplerEan-Fracttl,, 2022) for 80k steps with validation every 5k steps. Evaluation covers Visual Matching (VM) and Visual Aggregation (VA) settings. VM mirrors the real setup reduce sim-to-real gap, while VA stress-tests robustness by altering background, lighting, distractors, and table textures. The Fractal testbed includes 336 variants, yielding 2,856 trials in total.
evaluate on two SimplerEnv suites: Bridge and Fractal. For
train the RT-1 dataset (Brohan al.,
Evaluation Results on SimplerEny-Bridge As shown in Tab. 1, Memory VLA achieves aver- age success rate of 71.9%, a +1..6 point gain over the ACT-LLLe baseline, and surpasses recent state-of-the-art VLAs including To (Black et al., 2024). Per task, success rates 75.0% on Spoon on Towel, 75.0% on Carrot on Plate, 37.5% on Stack Cube, and 100.0% on Eggplant in Basket.
Evaluation Results SimplerEnv-Fractat Tab. reports results under Visual Matching and Visual Aggregation settings. VLA achieves an overall success rate of 72.7 %. improving CogACT by +4.6 points and surpassing To. By setting, the averages are 77.7% % on VM and 67.7 % on VA, gains of +2.9 and +6.4 points over CogACT, respectively. On Open/Close Drawer (VM), it reaches 84.7%, +12.9 point improvement over CogACT: under VA we observe larger gains, including +24.9 on Open/Close Drawer and +1 1.7 on Put in Drawer.
4.3 SIMULATED EVALUATION ON LIBERO
Training and Evaluation Setup We evaluate on the LIBERO (Liu et al., 2023a) benchmark with Franka robot across five suites: Spatial, Object, Goal, Long-10, and Long-90. The first four suites contain 10 tasks each, and Long-90 contains 90 tasks. Following Ope (Kim et al., 2024), 50 demonstrations per task are used. Separate models trained for Spatial, Object, and Goal for 20k steps each, while Long- 10 and Long-90 90 are trained jointly for 40k steps. Validation is performed

---

Published as a conference paper at ICLR 2026
Table Performance comparison SimplerEnv-Brice (Li et al., 2024b) with WidowX robot. CogACT-LLrge is our re-evaluated baseline using official weight, and Memory VLA achieves a +14.6 gain in average success. Entries marked with * are reproduced from open open-pi-zero, which leverage additional proprioceptive state inputs; they also adopt Uniform/Beta timestep sampling.
Spoon on Towel
Carrot on Plate
Stack Cube
Eggplant in Basket
Avg. Success
Method
RT-1-X (O'Neill al., 2024)
0.0
4.2
0.0
0.0
1.1
Open LA (Kim 2024)
4.2
0.0
0.0
12.5
4.2
Octo-Base
al., 2024)
15.8
12.5
0.0
41.7
17.5
TLA (Zheng et al. 2024b)
12.5
16.6
16.6
65.0
27.7
VLMS (Liu et al, 2025b)
45.8
20.8
4.2
79.2
37.5
SpatialVLA (Qu al. 2025)
16.7
25.0
29.2
100.0
42.7
Magma (Yang al., 2025a)
37.5
29.2
20.8
91.7
44.8
CogACT-Base (Li al, 2024a)
71.7
50.8
15.0
67.5
51.3
Ty- Uniform (Black et 2024)
63.3
58.8
21.3
79.2
55.7
CogACT-Large et. 2024a)
58.3
45.8
29.2
95.8
57.3
VLA (Li 2025a)
66.7
54.2
20.8
100.0
60.4
Ty-Beta* (Black et, 2024)
84.6
55.8
47.9
85.4
68.4
MemoryVLA VLA (Ours)
75.0
75.0
37.5
100.0
71.9
(+14.6)
Table 2: Performance comparison SinpaerEny-Fractall (Li et al., 2024b) with Google robot.
Success rates (%) are reported for Visual Matching ( (VM) and Visual Aggregation ( (VA) suites. Memo-
VLA achieves overall +4.6 gain over CogACT. O./C denotes Open/Close, and * follow Tab. 1.
Visual Matching (VM)
Visual Aggregation (VA)
Method
Coke Can
Move Near
O./C.
Put Drawer
Coke Can
Move Near
O./
Put in Drawer
Overall
Avg.
Avg.
Drawer
Drawer
ala 2014
22.7
1.2
6.1
RT-1-X at
56.7
31.7
59,7
21.3
42.4
49.0
32.3
29.4
10.1
30.2
36.3
OLe (Kierral 2004
18.0
56.3
63.0
0.0
34.3
67.7
28.8
0.0
39,3
36.8
RoboVLMs (Lix 2025))
76.3
79.0
44,9
27.8
57.0
50.7
62.5
10.3
0,0
30,9
44.0
TraceVLA (Zhengetal. 2014
45.0
63.8
63.1
11.1
45.8
64.3
60.6
61.6
12.5
49,8
47.8
RT-2-X al,
78.7
77.9
25.0
3.7
46.3
82.3
79.2
35.5
20,6
54.4
50.4
Magma (Yag etal, 20250
2125)
75.0
53.0
58.9
8.3
48.8
68.6
78.5
$9.0
24.0
57.5
$3.2
SpatialVLA (Querd, 2025)
79.3
54.6
56.0
78.7
83.0
518
$3.9
mo Uniform* (Black
2009)
88.0
80.3
$6.0
52.2
69.1
Reta" (ltcck otal., 2014
97.9
78.7
62.3
CogACT 2004)
2004))
91.3
85.0
71.8
50.9
74.8
89.6
80.8
28.3
46.6
61,3
68.1
MemoryVL. VLA (Ours)
90.7
88.0
84.7
47.2
77.7
80.5
78.8
53.2
58.3
67.7
72.7
(c4.6)
every 1k steps and results are reported at the best validation step. Each task is evaluated with 50 trials and per-suite average success rates are reported.
As shown in Tab. 3, MemoryVLA achieves an overall success rate of 96.5%, improving CogACT by +3.3 points and surpassing To. Per-suite success rates 98.4% on Spatial, 98.4% on Object, 96.4% on Goal, 93.4% on Long-10, and 95.6% on Long-90. Note that MemoryVLA uses only third-person RGB, without wrist views or proprioceptive states.
Evaluation Results on LIBERO
4.4 SIMULATED EVALUATION ON MIKASA-ROMA
Training and Evaluation Setup We evaluate on the Mikasa-Robo (Cherepanov et al., 2025) benchmark. our experiments, we compare MemoryVLA with VLA-style baselines used in Mikasa-Robo, including VLA-FFF (Kim al., 2025), PI-O (Black et al., 2024), and Spa- tialVLA (Qu et al., 2025). We additionally reproduce Cronus (Li et al., 2025a), a contem- poraneous VLA model that uses temporal context. Following the standard Mikasa-Ra Robo protocol using 250 demonstrations per task at 128x128 resolution, 100 evaluation episodes per task and end- effector control. Standard 5 tasks trained jointly for 20k steps, and validation is performed every 1 steps and results are reported at the best validation step.
8

---

Published as a conference paper at ICLR 2026
Table Performance comparison LIBERO (Liu al., 2023a) with Franka robot. Success rates (%) are reported across five suites. indicates methods using additional proprioceptive and wrist-camera inputs. CogACT results are reproduced by us. For methods without LIBERO-90 results, we report the average over the first four suites.
Method
Spatial
Object
Goal
Long-10 10
Long-90
Avg. Success
Diffusion Policy (Chi et al. 2023)
78.3
92.5
68.3
50.5
72.4
Octo (Team et 202)))
78.9
85.7
84.6
51.1
75.1
MDT (Renss et al., 2024)
78.5
87.5
73.5
64.8
76.1
UniACT (Zhong et al., 2025b)
77.0
87.0
77.0
70.0
73.0
76.8
Mall (Jia
74.3
90.1
81.8
78.6
83.5
Spatial (Qu 2025)
88.2
89.9
78.6
55.5
46.2
71.7
Trace VLA (Zheng et al., 2024b)
84.6
85.2
75.1
54.1
74.8
Open VLA (Kim al., 2024)
84.7
88.4
79.2
53.7
73.5
75.9
CoT-VLA (Zhao al, 2025)
87.5
91.6
87.6
69.0
81.1
To- FA-FAST* (Pertsch et al., 2025)
96.4
96.8
88.6
60.2
83.1
85.0
CronusVL.
2025a)
90.1
94.7
91.3
68.7
86.2
TriVLA (Liu et al., 2025d)
91.2
93.8
89.8
73.2
87.0
4D-VLA (Zhang et al,, 2025a)
93.8
92.8
95.6
86.5
92.2
CogACT
2024a)
97.2
98.0
90.2
88.8
92.1
93.2
To (Black et 2024)
96.8
98.8
95.8
85.2
94.2
Memory VT.A (Ours)
98.4
98.4
96.4
93.4
95.6
96.5 (43.3)
Table 4: Performance comparison on Mikasa-Robo (Cherepanov al., 2025) with Franka robot. Success rates (%) are reported. Cronus results are reproduced by us.
Model
ShellGame Touch
Intercept
Remb. Color3
Remb.
Remb. Color9
Avg. Success
Medium
Color5
Cronus VLA (Li et 2025a)
32
5
31
13
9
18.0
SpatialVLA (Qu et al,, 2025)
23
17
11
21.0
Open OLA-VLA-OFT (Kim 2025)
47
14
59
16
6
28.4
PI-O (Black 2024)
33
42
35
22
15
29.4
Memory VLA (Ours)
88
24
44
30
20
41.2 (+11.8)
Evaluation Results Mikasa-Robh MemoryVLA consistently achieves the highest perfor- mance across all tasks, with an average of 11.8% % improvement over the previous state-of-the-arr, especially +41.0% % ShellGame task. The results shown in Tab. 4.
4.5 REAL-WORLD EVALUATION
Training and Evaluation Setup We evaluate two real-robot suites, General and Long-horizon Temporal, on Franka and WidowX robots. Both use an Intel RealSense D435 RGB camera mounted in fixed front view. Images are captured at 640 x 480 and downsampled to 224 x 224. The system is integrated via ROS. For General, each task uses 50-150 demonstrations and evaluated from randomized initial states. Pick Diverse Fruits comprises five variants with 5 trials per variant (25 total); all other General tasks use 15 trials. For Long-horizon Temporal, each task uses 200-300 demonstrations and is evaluated with 10-15 trials using step-wise scoring reflect progress sub-goals. Training runs for approximately 5k-20k steps depending on the task and data size.
Evaluation Results on Real-world As shown in Tab. 5, Memory VLA achieves average success scores of 85% on general tasks and 83% on long-horizen temporal tasks, exceeding CogACT by +9 and +26 percentage points, respectively, and surpassing सo across both suites. On general tasks, it exceeds the strongest baseline every task, with notable gains Egg in Pan (+13) and Egg in Oven (+20), On long-horizon temporal tasks, improvements are larger, including +43 on Seq. Push Buttons, +38 on Change Food, +32 Guess Where, and +17 Clean Table & Count. These results demonstrate strong real-world competence and highlight the benefits of temporal memory.

---

Published as a conference paper at ICLR 2026
Table 5: Performance comparison on real-world experiments with Franka and WidowX robots. Success scores (%) are reported over six general tasks and six long-horizon temporal tasks. All methods are evaluated with only third-person RGB observation and language instruction.
General Tasks
Method
Insert Circle
Egg in Pan
Egg in Oven
Stack Cups
Stack
Pick Diverse Fruits
Avg. Success
Blocks
VLA (Kim et al,, 2004)
47
31
Ta (Black et 2024)
67
ACT (1á 20244)
80
67
93
MemoryVL.. VLA (Ours)
87
80
93
(+9)
Long-horizon Temporal Tasks
Method
Seq. Push
Change Food
Guess Where
Clean Table
Pick Place Order
Clean Rest. Table
Avg.
Buttons
& Count
Success
VLA (Kim 2004)
6
0
15
27
na 2024)
25
42
24
61
82
80
52
CogACT al.,
15
47
40
67
90
84
57
Memory (Ours)
58
84
100
83 (+26)
Table 6: Ablation memory type and length. We report average success rates (%) on SimplerEnn-Bridge tasks.
Table 7: Ablation memory retrieval, fu- sion, consolidation. We report average suc- cess rates (%) SimplerEnn--rridee tasks.
Avg. Success
Avg.
Variant
Variant
Success
Memory Type
Cognitive Mem. Perceptual Mem.
63.5
Retrieval
w/o Timestep PF
w/ Timestop PE
71.9
Both
71.9
Add
Fusion
67.7
Gate
71.9
Memory
Length
16
71.9
Consolidation
FIFO 66.7
64
67.7
Token Merge
71.9
4.6 ABLATION STUDIES
We ablate memory design SimplerEnv-Bridge quantify each choice. As shown in Tab. 6, combining perceptual and cognitive memory attains 71.9%, compared with 63.5% for cognitive- only and 64.6% for perceptual-only. A memory length of performs best at 71.9%, whereas 4 and 64 drop to 67.7%. Tab. 7 evaluates retrieval, fusion, and consolidation. Adding timestep positional encoding increases performance from 69.8% 71.9%. Gate fusion reaches 71.9%, compared with 67.7% for simple addition. Token-merge consolidation achieves 71.9% versus 66.7% with FIFO.
5 CONCLUSION
Inspired by cognitive science, we propose MemoryVLA, a Cognition-Memory--Acii framework for robotic manipulation. uses a hippocampus-like Perceptual-Cogniiive Memory Bank that coop- erates with working memory to capture temporal dependencies. VLM commonsense priors further support high-level cognition, while a rally aware actions. Across 150+ tasks with 500+ variations on 3 robots spanning SimplerEnv, LIBERO, and real-world, MemoryVLA consistently surpasses CogACT and To, achieves state- of-the-art performance, with notable gains challenging long-horizon temporal tasks. It also demonstrates strong robustness and generalization under diverse OOD conditions. Future direc- tions include (i) developing memory reflection, aligning long-term memory the LLM input space to enable embedding-spiie space chain-of of souin-of-thouseee thoughh reasoning; and (ii) building lifelong memory through biologically inspired consolidation that distills frequently reused experiences into permanent repre- sentations, thereby supporting scalable generalization across scenes, tasks, and embodiments.
diffusion action expert generates tempo-
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Science and Technology Major Project China un- der Grant No. 20232D0121300, the National Natural Science Foundation of China under Grants U24B20173 and 62321005, the Scientific Research Innovation Capability Support Project for Young Faculty under Grant ZYGXQNISKYCKN.ZCXM222
REPRODUCIBILIIY STATEMENT
The main paper specifies the model architectures, training setups, and experimental protocols, and the appendix provides additional details on hyperparameters, datasets, preprocessing and data aug- mentation, training procedures, evaluation methods and scoring rules, robustness and generalization analyses, task specifications, and qualitative results.. For full reproducibility, we release:
Models: http://ssspppptttt..coooooooooca
Logs: httpa//ss/agggggocccooooooooooea
Robotic videos: http:://tteeeeeaalalloo.../eeeen

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303, 087704, 2023.
Alan D Baddeley and Graham James Hitch. Working memory (vol. 8). New York: GA Bower (ed), Recent advances in learning and motivation, 1974.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li. Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chenggiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang. Shengguang Wu, Benfeng Xu, Jin Xu, An Yang. Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023a.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 2023b.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. pi-0: A vision- vision-language-aee model for general robot control. arXiv preprint arXiv:2410.24164. 2024.
flow
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212..68117 2022.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choroman- ski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-attion models transfer web knowledge to robotic control. arXiv preprint arXiv:2307. 2023.
Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.066 250666669, 2025a.
Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Univla: Learning act anywhere with task-centric latent actions. arXiv preprint arXiv:2505.06111, 2022..
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEFF/CVF international conference on computer vision, pp. 9650-9660, 2021.
Egor Cherepanov, Nikita Kachaev, Alexey K Kovalev, and Aleksandr I Panov. Memory. benchmark & robots: A benchmark for solving complex tasks with reinforcement learning. arXiv preprint arXiv:2502. 10550, 2025.
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, pp. 02783649241273668 2023.
Cunxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, et al. Interlcave-vla: Enhancing robot manipulation with interleaved image-text instructions. arXiv preprint arXiv:2505.02112, 2025.
Tingliang Feng, Hao Shi, Xueyang Liu, Wei Feng, Liang Wan, Yanlin Zhou, and Di Lin. Open com- pound domain adaptation with object style compensation for semantic segmentation. Advances in Neural Information Processing Systems, 36:631366 11131136-3143 2023.
Chongkai Gao, Zixuan Liu, Zhenghao Chi, Junshan Huang, Xin Fei, Yiwen Hou, Yuxuan Zhang, Yudi Lin, Zhirui Fang, Zeyu Jiang, et al. Vla-os: Structuring and dissecting planning representa- tions and paradigms in vision-language-action models. arXiv preprint arXiv:2506. /7561, 2025.
12

---

Published as a conference paper at ICLR 2026
Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Wei Chao, and Dieter Fox. Rvt: Robotic view transformer for 3d object manipulation. In Conference on Robot Learning, pp. 694-710. PMLR, 2023.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint (rXiv:2501.129444 2025.
Jonathan Ho and Tim Salimans. arXiv:2207.12598, 2022.
Classifier-free diffusion guidance.
arXiv preprint
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitatie networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132-7141, 2018
Xiaogang Jia, Qian Wang, Atalay Donat, Bowen Xing, Ge Li, Hongyi Zhou, Onur Celik, Denis Blessing, Rudolf Lioutikov, and Gerhard Neumann. Mail: Improving imitation learning with selective state space models. In 8th Annual Conference on Robot Leurning, 2024.
Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vims: Investigating the design space of visually-conditioned language models. In Forty-first International Conference Machine Learning. 2024.
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty El- lis, al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403. 12945, 2024.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source Assionllagguage-actoon model. arXiv preprint arXiv:2406.0922 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Opti- mizing speed and success. arXiv preprint arXiv:2502. 19645, 2025.
Hao Li, Shuai Yang, Yilun Chen, Yang Tian, Xiaoda Yang, Xinyi Chen, Hanqing Wang, Tai Wang. Feng Zhao, Dahua Lin, et al. Cronusvla: Transferring latent motion across time for multi-frame prediction in manipulation. arXiv preprint
2tt.::5506.188816 2025a.
Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, al. Cogact: A foundational lision-language-ace model for syn- ergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411. 1965000,, 2024a.
Runhao Li, Wenkai Guo, Zhenyu Wu, Changyuan Wang, Haoyuan Deng, Zhenyu Weng, Yap-Peng Tan, and Ziwei Wang. Map-vla: Memory-augmented prompting for vision-language-action model in robotic manipulation. arXiv preprini arXiv:2511 I. 1.0516, 2025b.
Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision-language foundation models as effective robot imitators. arXiv preprint 1xXiv2231.001378 2023.
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, et Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 20244.
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776-44791 2023a.
Chenghao Liu, Jiachen Zhang, Chengxuan Li, Zhimu Zhou, Shixin Wu, Songfang Huang, and Huil- ing Duan. Ttf-vla: Temporal token fusion via pixel-attention integration for langue-lan action models. arXiv preprint a50iv /9257, 2025a.
13

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 000444992-49166 2023b.
Huaping Liu, Xinghang Li, Peiyan Li, Minghuan Liu, Dong Wang, Jirong Liu, Bingyi Kang, Xiao Ma, Tao Kong, and Hanbo Zhang. Towards generalist robot policies: What matters in building vision- aiooo-llnguage-acten
models. arXiv preprint arXiv:2412. 14058, 2025b.
Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, et al. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-actinnn model. arXiv preprint arXiv:2503. 10631, 2025c.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang. Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pp. 38-55. Springer, 2024a.
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt- lb: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864 2024b.
Yingfei Liu, Junjie Yan, Fan Jia, Shuailin Li, Aqi Gao, Tiancai Wang, and Xiangyu Zhang. Petrv2: A unified framework for 3d perception from multi-camera images. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3262-3272, 2023c.
Zhenyang Liu, Yongchong Gu, Sixiao Zheng, Xiangyang Xue, and Yanwei Fu. Trivla: A uni- fied triple-system-bar based unified vision-language-atiion model for general robot control. arXiv preprint arXiv:2507.00 00..000..44,,,, 2025d.
Oier Mees, Dibya Ghosh, Karl Pertsch, Kevin Black, Homer Rich Walke, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, et al. Octo: An open-source generalist robot policy. In First Workshop on Vision- Vasion-Language Models for Navigation and Manipulation at ICRA 2024, 2024.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07/93, 2023.
Abby O' Neill,, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 6892-6903. IEEE, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205. 2005.
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025.
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang. Jia Yuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual- language-action model. arXiv preprint arXiv:2501.15830 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PmLR, 2021.
Moritz Reuss, Ömer Erdinç Yağmurlu, Fabian Wenzel, and Rudolf Lioutikov. Multimodal diffusion transformer: Learning versatile behavior from multimodal goals. arXiv preprint arXiv:2407.05996, 2024.
Valerie Reyna and Charles Brainerd. Fuzzy-trace theory: An interim synthesis. Learning and individual Differences, 7(1):1-75, 1995.
14

---

Published as a conference paper at ICLR 2026
Hao Shi, Bin Xie, Yingfei Liu, Yang Yue, Tiancai Wang. Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Spatialactor: Exploring disentangled spatial representations for robust robotic manipula- tion. arXiv preprint arXhv:2511.09555 2025.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robol Learning, pp. 785-799. PMLR, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 2020.
Lin Sun, Bin Xie, Yingfei Liu, Hao Shi, Tiancai Wang, and Jiale Cao. Geovla: Empowering 3d representations in ssioo-language-action models. arXiv preprint arXiv:2508.0907) 2025.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405. /2213, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprini vrXiv:2307.09288 2023.
Endel Tulving et al. Episodic and semantic memory. Organization memory, 1(381-403):1, 1972.
Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning scale. In Conference Robot Learning, pp. 1723-1736. PMLR, 2023.
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. Proceedings of the IEEF/CVF international conference on computer vision, pp. 3621-3631, 2023.
Yulin Wang, Yang Yue, Yang Yue, Huanqian Wang, Haojun Jiang, Yizeng Han, Zanlin Ni, Yifan Pu, Minglei Shi, Rui Lu, et al. Emulating human-like adaptive vision for efficient and flexible machine visual perception. Nature Machine Intelligence, pp. 1-19, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837 2022.
Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. Dexvla: Vision-languaae model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.05855, 2025.
Dongming Wu, Yanping Fu, Saike Huang. Yingfei Liu, Fan Jia, Nian Liu, Feng Dai, Tiancai Wang, Rao Muhammad Anwer, Fahad Shahbaz Khan, et al. Ragnet: Large-scale reasoning- based af- fordance segmentation benchmark towards general grasping. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11980-11999 2025.
Bin Xie, Erjin Zhou, Fan Jia, Hao Shi, Haoqiang Fan, Haowei Zhang, Hebei Li, Jianjian Sun, Jie Bin, Junwen Huang, et al. Dexbotic: Open-source vision- assion-lagguagc-attion toolbox. arXiv preprint arXiv:2510.23511 2025.
Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijic Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, et al. Magma: A foundation model for multimodal ai agents. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 14203-14214, 2025a.
Shuai Yang, Hao Li, Yilun Chen, Bin Wang, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, Yiyi Liao, and Jiangmiao Pang. Instructvla: Vision-language-actoon instruction tuning from under- standing to manipulation. arXiv preprint arXiv:2507.17522 2025b.
Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang, Shiji Song, Jiashi Feng, and Gao Huang. Decr-vla: Dynamic inference of multimodal large language models for efficient robot execution. Advances in Neural Information Processing Systems, 14556699-66644 2024.
15

---

Published as a conference paper at ICLR 2026
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
Jiahui Zhang, Yurui Chen, Yueming Xu, Ze Huang, Yanpeng Zhou, Yu-Jie Yuan, Xinyue Cai, Guowei Huang, Xingyue Quan, Hang Xu, et al. 4d-vla: Spatiotemporal vision-language-action pretraining with cross-scene calibration. arXiv preprint arXiv:2506.22244 2025a.
Yani Zhang, Dongming Wu, Hao Shi, Yingfei Liu, Tiancai Wang, Haoqiang Fan, and Xingping
Dong. Grounding beyond detection: Enhancing contextual understanding in embodied 3d ground- ing. arXiv preprint arXiv:2506.0 1999, 2025b.
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, al. Cot-vla: Visual chain-of- reasoning for vision- language-action models. In Proceedings of the Computer Vision and Pattern Recognition Confer- ence, pp. 1702-1713, 2025.
Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine- grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304. 13705, 2023.
Henry Zheng, Hao Shi, Yong Xien Chng, Rui Huang, Zanlin Ni, Tianyi Tan, Qihang Peng, Yepeng Weng, Zhongchao Shi, and Gao Huang. Denseg: Alleviating vision-language feature sparsity in multi-view 3d visual grounding. In Autonomous Grand Challenge CVPR 2024 Workshop, volume 2. pp. 6, 2024a.
Henry Zheng, Hao Shi, Qihang Peng, Yong Xien Chng, Rui Huang, Yepeng Weng, zhongchao shi, and Gao Huang. Densegrounding: Improving dense language-vision semantics for ego-centric 3d visual grounding. In The Thirteenth International Conference on Learning Representations, 2025a.
Jinliang Zheng, Jianxiong Li, Dongxiu Liu, Yinan Zheng, Zhihao Wang. Zhonghong Ou, Yu Liu, Jingjing Liu, Ya-Qin Zhang, and Xianyuan Zhan. Universal actions for enhanced embodied foun- dation models. Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 22508 225199 2025b.
Ruijie Zheng, Yongyuan Liang Shuaiyi Huang, Jianfeng Gao. Hal Daumé III, Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint arXiv:2412 2024b.
Yifan Zhong, Fengshuo Bai, Shaofei Cai, Xuchuan Huang, Zhang Chen, Xiaowei Zhang, Yuanfei Wang. Shaoyang Guo. Tianrui Guan, Ka Nam Lui, et al. A survey vision-language-attio models: An action tokenization perspective. arXiv preprint arXiw:2507.01925, 2025.
Junhao Zhou, Ziqi Pang, and Yu-Xiong Wang. Rmem: Restricted memory banks improve video ob- ject segmentation. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 18602-18611, 2024.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Iision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pp. 2165-2183. PMLR, 2023.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A LLM Usage
18
Robustness and Generalization Evaluation
18
B.1 Real-world Evaluation
18
B.2 Simulation Evaluation..
18
Additional Training Details
19
C.1 Hyper-parameters
19
C.2 Training Data
20
C33 Training Setup...
22
C.4 Data Augmentation
22
D Additional Evaluation Details
23
D.I SimplerEnv
23
D.2 LIBERO
23
D.3 Mikasa-Robo
23
D.4 Real-world
23
Case Study of Memory Retrieval
24
F Data Length Statistics
25
G Additional Ablation Study
25
H Inference Efficiency
26
I Zero-shot Task Generalization
27
J Compare with Temporal Context VLA Baselines
27
K Task Details
27
K.1 Real-world Tasks
29
................................
K.2 SimplerEnv Tasks
................................
29
K.3 LIBERO Tasks
30
Visualization of Memory Dependence in Real-World Tasks
30
M Qualitative Results
32
M.1 Real-world Evaluation
32
M.2 Simulation Evaluation
33
17

---

Published as a conference paper at ICLR 2026
A LLM USAGE
Large Language Models (LLMs) were used only for grammar and wording polishing. All research content and contributions are entirely the responsibility of the authors, with involvement of LLMs.
B ROBUSTNESS AND GENERALIZATION EVALUATION
B. REAL-WORLD EVALUATION
(a) Various OOD Variants Pick Place Order
Unseen
Unseen Distractors
Unseen Lightnn
Unseen Object
Unseen Container
Base
Occlusion
Background
(b) Various OOD Variants of Clean Restaurant Table
Unseen Background
Unseen Distructors
Unseen Lighting
Unseen Object
Unseen Container
Occlusion
Generalization of Pick Place Order
(d) Generalization of Clean Restaurant Table
90 92
100
96
100
96
96
94
96
94
89
90
86
80
80
70
Lishtng
Backgondd
Lightng
Figure 5: Robustness and generalization under out-of-distribution (OOD) conditions real- world. (a.b) Examples of OOD variants for two representative tasks (Pick Place Order and Clean Restaurant Table), including unseen backgrounds, distractors, lighting, novel objects/cconaainers, and occlusion. (c,d) Quantitative results showing that MemoryVLL VLA maintains high success rates across these OOD variants, demonstrating strong robustness and generalization in real-world environments.
We further assess the robustness and generalization of Memory VLA in real-world environments un- der diverse out-of-distribution (OOD) variants. Fig. 5 shows two representative tasks, Pick Place Order and Clean Restaurant Table, evaluated under unseen backgrounds, distractors, novel ob- jects/containers. lighting variations, and occlusions.
For Pick Place Order, MemoryVLA attains near wear-perfect success under the base setting (100%), unseen background (100%), unseen distractors (92%), unseen lighting (96%), unseen container (100%), and occlusion (96%), with moderate drop unseen objects (89%). For Clean Restau- rant Table, the base success rate is 96%, with unseen background (92%), unseen distractors (86%), unseen lighting (94%), unseen object (94%), unseen container (96%), and occlusion (94%).
These results confirm that MemoryVLA maintains consistently high performance across a wide range of real-world OOD conditions, demonstrating strong robustness and generalization.
B.2 SIMULATION EVALUATION
We further conduct robustness and generalization experiments in simulation, considering both pick- and-move tasks and hinge-like object manipulation tasks. Fig. presents results on Pick Coke Can
18

---

Published as a conference paper at ICLR 2026
(a) Various OOD Variants Pick Cole Can
Unseen Background
Unseen Background
Unseen Distractors
Unseen Distructors
Unseen Lighting
Unseen Lighting
Base
100.0
82.0
90.7
88.7
90.7
86.7
80.0
40.0
Unseen Camora View
Unseen
20.0
Umezn Datraciion
Ureeze Barkgrand
Iheam Lightnn
Unson Coman View
Unwren Tranre
Camern View
(b) Various OOD Variants of Move Near
Unseen Background
Unseen Background
Unseen Lighting
Unseen Lighting
Unseen Camera View
Base
Distracters
100.0
86.0
86.0
76.0
80.0
60.0
Unseen
Unseen Texture
Unseen
20,0
Basc
Lieazz
Unaan
Unsem
Unaczn
Useaz Testure
Camera View
Texture
Dianacur
Backgound
Lighing
Camera View
Figure 6: Robustness and generalization under out-of-distribune
(OOD) variants in simula-
tion: Pick and Move tasks. (a) Pick Coke Can and (b) Move Near tasks evaluated under unseen backgrounds, distractors, lighting. textures, and camera views. Bar plots report the corresponding success rates, showing that Memory VLA maintains strong performance across most shifts, with the largest degradation under unseen camera views.
and Move Near, while Fig. covers Open/Close Drawer and Place Apple Into Drawer. These tasks are evaluated under unseen backgrounds, distractors, lighting, textures, and camera views.
For Pick Coke Can, MLA. achieves base success of 92.0%, with unseen distractors (90.7%), unseen background (86.7%), unseen lighting (90.7%), and unseen texture (86.7%), while performance drops substantially under unseen camera views (42.0%). For Move Near, the base suc- cess rate is 76.0%, with unseen distractors (84.0%), unseen background (86.0%), unseen lighting (84.0%), unseen camera view (58.0%), and unseen texture (86.0%). For hinge-like object manipula- tion, Open/Close Drawer yields base success rate of 46.3%, unseen background (56.4%) unseen lighting (49.1%) and unseen texture (57.4%). For Place Apple Into Drawer, the base success rate is 72.0%, with unseen background (66.0%) unseen lighting (52.0%), and unseen texture (50.0%).
These results show that MemoryVI.A generalizes well across moderate distribution shifts such as distractors, backgrounds, and textures, but suffers more under severe changes, especially unseen camera views.
C ADDITIONAL TRAINING DETAILS
C.I EYPER-PARAMETERS
We summarize the main hyperparameters used in experiments. The global batch size is 256 (32 x GPUs), the learning rate is x 10-5, and gradients are clipped at a max norm of 1.0. The policy predicts 16-step action chunk, and perceptual tokens use 256 channels. The diffusion policy
19

---

Published as a conference paper at ICLR 2026
Various OOD Variants of Open Closed Drawer
Unseen Background
Unseen Background
Unseen Lighting
Unseen Lighting
Unseen Texture
Unseen Texture
Base
Various OOD Variants of Place Apple Into Drawer
Unseen Background
Unseen Background
Unseen Lighting
Unseen Lighting
Unseen Texture
Unseen Texture
(c) Generalization of Open/ / Closed Drawer
(d) Generalization of Place Apple Into Drawer
49.1
66.0
45.0
60.0
52.0
50.0
30.0
40.0
15.0
20.0
Base
Unseen
Unseen
Unseen
Base
Unseen
Unseen Lighting
Unseen Texture
Background
Lighting
Texture
Background
Figure 7: Robustness and generalization under tittof--ditribbution (OOD) variants in simu- Hinge-like object manipulation. (a) OOD variants of Open/Close Drawer and (b) Place Apple Into Drawer tasks, including unseen backgrounds, distractors, lighting, textures, and camera views. Quantitative results indicate that Memory VLA generalizes well under moderate shifts, while performance drops notably with camera views changes.
lation:
Table 8: Training and model hyperparameters.
Hyperparameter
Value
Batch size
32 x 8
Learning rate
10
Repeated diffusion steps
4
Action trunking size
16
Perceptual token channels
256
Max grad. norm
1.0
CFG scale (classifier-free guidance)
1.5
uses 4 repeated diffusion steps during training: inference uses DDIM with 10 sampling steps and classifier-free guidance scale of 1.5. See Tab. 8 for a concise summary.
C.2 TRAINING DATA
Bridge v2 For the SimplerEnv-Briee benchmark, we train on BridgeData v2 (Walke et al., 2023), large-scale, language-conditiones real-robot manipulation dataset of roughly 60,000 teleoperated trajectories collected on WidowX robots across diverse tabletop settings. Episodes pair language instructions with demonstrations of skills such as picking, placing, pushing, stacking, and folding.
RT-1 For the SimplerEnv-Freee Fractal benchmark, use RT-1 (Brohan et al., 2022), large-scale real-world dataset of roughly 130,000 episodes spanning 700+ tasks, collected over 17 months by the Google Robot fleet and paired with natural-language instructions.
20

---

Published as a conference paper at ICLR 2026
Franka Robot
Intel Realsense D435 Camera
Figure 8: Franka robot setup.
Intel Realsense D435 Camera
WidowX Robot
Figure 9: WidowX robot setup.
LIBERO LIBERO (Liu et al., 2023a) provides simulation tasks with a Franka robot across five suites: Spatial, Object, Goal, Long-10, 10, and Long-90, totaling 130 language-conditioned tasks. Each task supplies 50 demonstrations.
Mikasa-Robh Mikasa-Robo (Cherepanov et al., 2025) comprises five memory-dependent manip- ulation tasks, each with 250 officially provided demonstrations, using A end-effector control.
21

---

Published as a conference paper at ICLR 2026
Real-world We collect real demonstrations Franka and WidowX robots using fixed third- person RGB setup, as shown in Fig. 8 and 9. A front-facing Intel RealSense D435 captures 640 x 480 RGB at 30 fps. Franka uses single end-effector per experiment, either the stock parallel gripper or a Robotiq parallel gripper. Demonstrations are gathered by joystick teleoperation. The General suite uses 50-150 demonstrations per task, and the Long-horizon Temporal suite uses 200-300 per task. The system is integrated in ROS.
After collection, we perform a standardized preprocessing pipeline. Frames are downsampled to 224 x 224. We then subsample the video stream by retaining a frame whenever the end-effector translation since the last kept frame exceeds 0.01 or the orientation change exceeds radrad, and we also enforce a maximum gap of 120 frames between kept frames. The processed episodes are converted into the RLDS format for downstream training.
C.3 TRAINING SETUP
SimplerEnv-Bridee On Bridge v2, models are trained for 50k steps with stream dataloader. Each episode is unpacked into consecutive frames tagged with its episode ID. During training, batches are filled sequentially with frames from single episode whenever possible. If an episode ends before the batch is complete, the remaining slots are filled with frames from the following episode. A new batch then continues from the position where the previous one stopped, ensuring that in-episode temporal order is always preserved. The memory length is fixed to 16.
SimplerEnv-Fractal Models are trained for 80k steps on RT-1 The benchmark defines two pro- tocols: Visual Matching (VM), which mirrors the real-robot setup, and Visual Aggregation (VA), which perturbs background, lighting, distractors, and textures to test robustness. The dataloader design and memory length follow the same setup in CemplerEnv-Bridee
LIBERO Following VLe (Kim et al., 2024), we train with 50 demonstrations per task after removing failed trajectories from the dataset. Spatial, Object, and Goal suites are trained separately for 20k steps each, while Long-10 and Long-90 are treated as single family of long-horizon data and trained jointly for 40k steps. The dataloader adopts grouped sampling strategy: in each iter- ation, 16 frames are randomly sampled from within a single episode, matching the memory length of 16 used throughout training. The dataloader adopts a grouped sampling strategy in which each batch is divided into multiple groups, and each group consists of several frames drawn from single episode. Frames within a group are kept in temporal order. The memory length is set to 16.
Mikasa-Robo Following Mikasa-Robe (Cherepanov et al., 2025), adopt the standard protocol with five tasks and train jointly on all 1,250 demonstrations for 20k steps, using 128 x 128 RGB observations and A end-effector control. We reuse the same dataloader setup as in LIBERO, and set the memory length to 16.
Real-world Models are trained for 5k-20k steps depending on task and dataset size. The gen- eral tasks contain 50-150 demonstrations per task, while long-horizon temporal tasks use 200-300 demonstrations per task. The memory length is set to 16 for general tasks and 256 for long-horizon temporal tasks.
C.4 DATA AUGMENTATION
We apply standard fer-frame augmentations to the third-person RGB stream during training. Aug- mentations are applied in a fixed order: random resized crop, random brightness, random contrast, random saturation, and random hue. The crop samples 90% of the image area with aspect ratio 1.0 and resizes to 224 224. Brightness is perturbed with magnitude 0.2, contrast and saturation are scaled in [0.8, 1.2], and hue is shifted by up to 0.05. All augmentations are disabled at evaluation.
22

---

Published as a conference paper at ICLR 2026
D ADDITIONAL EVALUATION DETAILS
D.1 SIMPLERENV
Evaluation follows the official CogACT protocol (Li et al., 2024a). We adopt the same evaluation scripts and use the adaptive action ensemble strategy introduced in CogACT, with ensemble coeffi- cient 0.1, ensemble horizon set 7 for Bridge and for Fractal. For Bridge, models are trained for 50k steps and validated every 2.5k steps, since the denoising objective of diffusion models does not reliably indicate policy quality, we report success rates at the best validation step. For Fractal, training runs for 80k steps with validation every 5k steps, and evaluation covers 336 variants in to- tal (Tab, 19), we similarly report success rates at the best validation step, VM and VA settings are evaluated separately.
Since the original paper only reported per-task success rates for CogACT-Bas but not for CogACT- Large, re-evaluated the released CogACT-Large checkpoint in our setup and report those num- bers for fairness. For To, results are taken from the open- source reproduction open-pi-zent which provides implementations with both uniform and beta timestep sampling strategies in flow match- ing. We report results under float32 precision as in the public release. Note that open-pi-zero does not provide numbers for the Fractal Visual Aggregation setting, and thus these are missing.
D.2 LIBERO
Evaluation on LIBERO (Liu et al., 2023a) is conducted across all five suites (Spatial, Object, Goal, Long-10, and Long-90). Models are validated every 1k steps, and each task is evaluated with 50 trials. Success rates reported at the best validation step. Unlike SimplerEnv, no action ensemble strategy is used in our LIBERO experiments.
CogACT results reproduced using the official codebase for fairness. For To and mo FAST, adopt reported numbers, noting that both methods leverage additional wrist-camera views and pro- prioceptive states, while approach relies solely on single third-person RGB. Despite this differ- ence in input modalities, our method consistently surpasses, achieving stronger performance without extra sensory inputs.
D.3 MIKASA-ROBE
For Mikasa-Robo (Cherepanov et al., 2025), we evaluate with 100 episodes per task. Validation is performed every Ik training steps, and success rates are reported at the checkpoint with the best validation performance. As in LIBERO, we do not any action ensemble strategy in our Mikasa- Robo experiments.
D.4 REAL-WORLD
Evaluation uses 15-25 trials for General tasks and 10-15 trials for Long-horizon Temporal tasks. For General tasks, Pick Diverse Fruits contains five variants (apple, orange, banana, chili, grape), each evaluated with 5 trials (25 total). All other General tasks are evaluated with 15 trials each, and we report task-level success rates.
For horg-horizonn Temporal tasks, Seq. Push Buttons includes three button orders siiee-innkggeeree green, green-blue-pe blue--ikk-,- each tested with 5 trials. All other tasks are evaluated with 10 trials, and step-wise scoring is adopted to capture partial progress. The scoring rules are as follows:
Seq. Push Buttons: pressing each correct button yields 30, with bonus of 10 if all three are correct. Loose matching allowed (slight contact counts as a press).
Change Food: lifting and removing the initial food (30), grasping the new food (30), and placing it on the plate (30), with a 10 bonus for full success.
Guess Where: grasping the cover (30), covering the block (30), and uncovering it (40).
Clean Table Count: five objects in total. For each object, clearing yields 10 points and pressing the counter yields 10. Small counting errors (incomplete press / one extra press) earn 5; major errors (missed count multiple extras) earn O. Empty grasps with clear counting intent incur 5-point penalty.
23

---

Published as conference paper at ICLR 2026
Pick Place Order: carrot, banana, and orange must be picked and placed in sequence. Each correct step earns 30, with a 10 bonus for full completion. Any order violation terminates the attempt.
Clean Restaurant Table: five objects in total. Fach correctly sorted into trash bin or storage bin scores 20. Misplacement earns 10, and merely lifting without correct placement earns 5.
E CASE STUDY OF MEMORY RETRIEVAL
To provide direct view of how the memory mechanism functions, Fig. 10 visualizes the retrieved memory elements and their attention weights on the real-world and simulation tasks. The model consistently attends past frames that resolve decision-relevant ambiguities absent from the current observation.
In the real-world Change Food task, after the first food item is placed aside, the current frame con- tains two food items on the table, making it impossible to determine from this single observation which one should be picked next. RemooyVL.Ahhrrefeee
attends strongly to the nearby frames re- flecting the recent motion trend, as well as the last decisive frame before the ambiguity arises. In the Shell Game Touch task from Mikasa-Robo Simulation Benchmark, the robot briefly shown the cube location before it is covered by cups. The model consistently attends to the initial revealing frames, which provide the only reliable cue identifying strate that MeAnnryyLLrrrrriveees meaningful temporal cues essential for disambiguating the next action, rather than simply recalling redundant visual history.
correct cup. These results demon-
(a) Visualization of Memory Mechanism in World
Pick Up
Take Away
Confusion
Pick Another
Place. Another
Over!
CG
Move left or right?
Just begin or
almost over?
Moving from right to left
Allerion Score
Altention beare
Not just
placed on right side
Prompt
Move food off the plate, then put the other food an
44
20
0.5
150
(b) Visualization Memory Mechanism Simulation
Attuntion Store
Prompt
Memorize
Move Down
Reach
Intiaal pos
posilion of the cup covering then pick that cup
Case study of memory retrieval in real-world and simulated tasks. The figure vi- sualizes the retrieved memory elements and their attention weights on the real-world Change Food task (top) and the simulated Shell Game Touch task in Mikasa-Role (bottom). In both settings, the model consistently attends to past frames that resolve decision-relevan ambiguities absent from the current observation.
Figure 10:
24

---

Published as a conference paper at ICLR 2026
F DATA LENGTH STATISTICS
Tab. 9 reports the maximum, minimum, median, and average action lengths across all task suites, including SimplerEnv Evaluation (Bridge, Fractal), LIBERO (Spatial/OOjeet/Goall and 10/90 task suites), and both real-world general and temporal tasks. For the real-world tasks, we additionally provide filtered statistics based on a motion-magnitude threshold (translation > I 0.4 rad between consecutive frames) to remove frames where the end-effector motion is small.
rotation >
Table 9: Action Length Statistics across all simulation (SimplerEnv Bridge/Fractal. LIBERO Spa- tial/Object/Goal, LIBERO-10/90 10/90) and real-world (General, Temporal) task suites. For real-world tasks, the "Filtered" versions remove frames whose end-effector motion negligible (translation cm and rotation 0.4rad).
Task Suite
Max
Min
Median
Average
SimplerEnv-Bridge Fractal
200
80
117
119
LIBERO-Spatial Object Goal
270
75
131
130
LIBERO-10
505
58
144
156
Real-General (Original)
1575
281
575
575
Real-General (Filtered)
213
40
84
Real-Temporal (Original)
7704
412
981
1672
Real- Real-Tempael Telporal (Filtered)
902
72
236
288
G ADDITIONAL ABLATION STUDY
For memory length, it is closely tied to the episode length of each benchmark. As shown in Tab. 9, SimplerEnv, LIBERO, and our real-world general tasks have similar episode lengths, whereas real- world temporal tasks are substantially longer, even after heavy frame filtering. This naturally mo- tivates using larger memory length for temporal tasks. We further conducted an ablation on representative real-world temporal task (Clean Table & Count). A moderate memory length (256) performs best, as shown Tab. 10.
We also added an ablation on the number of cognitive tokens. As shown in Tab. 11, increasing the count from 1 to brings no performance gain. Perhaps the single 4096-dim EOS token already captures sufficient semantic information, so adding more tokens does not provide additional benefit.
In addition, we extended the ablations on fusion type (Tab. 12) and consolidation strategy (Tab. 13) to LIBERO-Long-90 and real-world long-horizon task.
Table 10: Additional ablation on memory length for both real-world and LIBERO-90 tasks.
(a) Real-World: Clean Table & Count
(b) LIBERO-Long-90 Tasks
Memory Length
Success Rate
Memory Length
Success Rate
64
78
8
94.2
256 (Base)
84
16 (Base)
95.6
512
81
32
95.6
Increasing the number of cognitive tokens from to does not improve performance. A single 4096-dim EOS token already provides sufficient semantic capacity.
Table 11: Ablation the Number of Cognitive Tokens.
Spoon on Towel
Carrot on Plate
Stack Cube
Eggplant in Basket
Num. Cog Token
Avg. Success
(Base)
75.0
75.0
37.5
100.0
71.9
79.2
66.7
37.5
95.8
69.8
25

---

Published as a conference paper at ICLR 2026
Table 12: Additional ablation on memory fusion type for both real-world and LIBERO-90.
Clean Table & Count (Real-World)
LIBERR-LLog-99 Tasks
Fusion Type
Add
78
93.8
Gate
84
95.6
Table 13: Additional ablation memory consolidation for both real-world and LIBERO-90.
Clean Table & Count (Real-World)
LIBERO-Long-90 Tasks
Consolidation Type
FIFO
76
94.9
Token Merge
84
95.6
Tab. 14 provides extended version of Tab. 6 and 7, reporting per-task success rates SimplerEnv- Bridge for all ablation settings. Gray rows indicate the default configuration.
Details of ablation studies. We report average success rates (%) on SimplerEnv-Brien when varying five factors: (a) memory type, (b) memory length, (c) memory retrieval, (d) memory fusion, and (e) memory consolidation. Gray rows indicate the default configuration.
Table 14:
Spoon on Towel
Carrot on Plate
Stack Cube
Eggplant in Basket
Avg.
Method
Success
Cog. Mcm.
70.8
58.3
29.2
95.8
63.5
(a) Memory Type
Per. Mem. Both
83.3
54.2
20.8
100.0
64.6
75.0
75.0
37.5
100.0
71.9
4
79.2
75.0
25.0
91.7
67.7
(b) Memory Length
16
75.0
75.0
37.5
100.0
71.9
64
79.2
54.2
37.5
100.0
67.7
w/o Timestep PE
83.3
62.5
50.0
83.3
69.8
(c) Memory Retrieval
w/ Timestep PE
75.0
75.0
37.5
100.0
71.9
Add
75.0
62.5
33.3
100.0
67.7
(d) Memory Fusion
Gate
75.0
75.0
37.5
100.0
71.9
FIFO
66.7
66.7
33.3
100.0
66.7
(c) Memory Update
Token Merge
75.0
75.0
37.5
100.0
71.9
H INFERENCE EFFICIENCY
As shown in Tab. 15, latency, throughput, and GPU memory measurements comparing our method with the baseline two commonly used GPUs, RTX 4090 and HGX H20. The results below averaged over 300 runs in b6f
Our method achieves latency of 0.194 and throughput of 82.5 Hz RTX 4090, corresponding to only a 3.6% increase in overhead compared to the baseline. The memory usage is 16.6 GB, which merely +0.8 GB over the baseline. These results show that the memory module lightweight and introduces only negligible extra cost during inference.
The overhead remains small because the memory module deliberately lightweight: each cognitive memory entry is represented by single cognitive token, the perceptual memory is compressed to 256 channels through perceptual compression module, and the memory consolidation step contin- uously merges redundant entries. Together, these designs keep both the retrieved memory size and cross-attention cost minimal, resulting in negligible additional latency and memory usage during inference.
26

---

Published as a conference paper at ICLR 2026
Table 15: Inference efficiency comparison. Latency, throughput, and GPU memory are measured over 300 runs in 6 with action chunk length set to 16 on RTX 4090 and HGX H20 GPU. MemoryVLA introduces only minor overhead compared the baseline.
Lateney (RTX 4090)
Throughput (RTX 4090)
Lateney (HGX H20)
Throughput (HGX H20)
Model
Memory
Baseline
0.187 0.194
85.6 Hz 82.5 Hz
0.236 0.246
67.8 Hz 65.0 Hz
15.8 GB
Memory VLA
16.6 GB
I ZERO-SHOT TASK GENERALIZATION
In addition visual OOD tests, have added task generalization experiments evaluate zero shot performance on unseen task categories. As shown in Tab. 16, We use Apple To Basket as the base task and test on three unseen tasks: Eggplant To Basket, Blush To Basket, and Apple To Plate. As shown in the table below, our method achieves good zero shot task generalization.
Table 16: Zero-shot task generalization results. Apple To Basket is used as the base task, and evaluation conducted on three unseen task categories.
Apple To
Eggplant To
Blush To
Apple To
Avg. Success (OOD Task)
Model
Basket (Base Task)
Basket (OOD Task)
Basket (OOD Task)
Plate (OOD Task)
Memory VLA
4/5
15
2/5
2 2/5
0.47
J COMPARE WITH TEMPORAL CONTEXT VLA BASELINES
As shown in Tab. 17, we have added temporal-context baselines for each benchmark, including Mikasa-Robo (Cherepanov et al., 2025), LIBERO (Liu al., 2023a), and SimplerEnv (Li et al., 2024b), to provide a fair and comprehensive comparison.
In the Mikasa-Robo benchmark, memory-dependent manipulation benchmark, no temporal- context VLA baselines are provided in the official release. To ensure a fair comparison, we there- fore reproduced Cronus (Li et al., 2025a), a contemporaneous strong VLA model that explic- itly leverages historical information by aggregating multi-frame VLM features through a sliding- window module. This provides Mikasa-Robh Robo with temporal-context baseline, and Memory VL- substantially outperforms it across all tasks.
For the LIBERO benchmark, we include the main temporal-contert VLA baselines commonly used in prior work, including TTF-VLA (Liu al., 2025a), TLA (Zheng al., 2024b), 4D- VLA (Zhang et al., 2025a), CronusVLA (Li et al., 2025a), and MAP-VLA (Li et al., 2025b). These methods represent the strongest publicly available temporal-context VLA models that have reported results on LIBERO. Memory VLAobtains higher average success rates than these baselines.
For the SimplerEnv benchmark, prior work, including VLA (Zheng et al., 2024b), RLbo (Liu et al., 2025b), and CronusVLA (Li et al., 2025a). These methods incorporate explicit temporal modeling, and Memo- LLAhhiveesess higher success rates than these baselines.
include the temporal-context VLA baselines reported in
K
TASK DETAILS
To ensure comprehensive evaluation across simulation and real-world settings, we summarize the task design of each benchmark. We provide task templates, variation types, and the number of variations per task to clarify the diversity and difficulty of evaluation.
27

---

Published as a conference paper at ICLR 2026
Table 17: Comparison with temporal-context VLA methods across diverse benchmarks. Mem- ory VLA outperforms temporal-context VLA baselines across both memory-focused benchmarks and general manipulation benchmarks, including Mikasa-Robo, SimplerEny, and LIBERO.
(a) SimplerEnv Benchmark
Spoon on Towel
Carrot Plate
Stack Cube
Eggplant in Basket
Avg.
Method
Success
Trace (Zheng et 2024b)
12.5
16.6
16.6
65.0
27.7
Robo VLMs (Liu al., 2025b)
45.8
20.8
4.2
79.2
37.5
CronusVLA (Ti at, 2025a)
66.7
54.2
20.8
100.0
60.4
Memory VLA (Ours)
75.0
75.0
37.5
100.0
71.9
(b) LIBERO Benchmark
Avg.
Method
Spatial
Object
Goal
Long-10 10
Success
TTF- VLA
2025a)
73.0
84.0
81.0
58.0
74.0
Trace VLA (Zheng et ol., 2024h)
84.6
85.2
75.1
54.1
74.8
4D-VLA (Zhang et al., 2025a)
88.9
95.2
90.9
79.1
88.6
Cronus al., 2025a)
93.8
92.8
95.6
86.5
92.2
MAP-VLA (Li al., 2025b)
96.3
98.4
95.4
83.4
93.4
Memory VLA (Ours)
98.4
98.4
96.4
93.4
96.7
(c) Mikasa-Robb Benchmark
Shell Game Touch
Intercept
Remb. Color3
Remb. Color5
Remb. Color9
Avg. Success
Model
Medium
Cronus VLA (Li et al., 2025a)
32
5
31
13
18.0
MemoryVLA VLA (Ours)
88
24
44
30
20
41.2
Table 18: Real-world tasks details. We list the instruction template, number of variations, and the corresponding variation types for each task.
Task Name
Language Instruction Template
# Variations
Variation Type
Seq Push Buttons
"Push the (color color 2, and color 3} buttons sequence"
3
Button color order
Change Food
"Move food off the plate, then put the other food
Food object type in plate
Guess Where
"Place the cover"
the block, then remove
Clean Table Count
"Clean the table item item, and push the button after each item cleaned" "Pick up carrot, bamana and orange in order and place them in the basket"
Pick Place Order
7
Background, distractors.
lighting. object, container, occlusion
Clean Restaurant Table
" Place all trash into the trash bin and all tableware into the storage bin"
7
Background, distractors, lighting, object, container, acclusion
Insert Circle
"Insert the circle on the square"
Egg Pan
"Put the egg into pan"
Egg In Oven
"Put the egg into the oven"
Stack Cup
"Stack the green cup on the other cup"
Stack Block
"Stack yellow block on block"
Pick Diverse Fruit
"Pick up (fruit) and place it hasket"
Fruit category
28

---

Published as a conference paper at ICLR 2026
Table 19: SimplerEnv tasks details. VM visual-matching, VA variant-agerrgaaton. For Fractal, we report VM and VA separately; all Bridge tasks use single setting.
Task Name
Language Instruction Template
Variations
Variation Type
Spoon On Tower
"Put the spoon tower"
Carrot On Plate
"Put the carrot
plate"
Bridge
Stack Cube
"Stack the green cube the yellow cube"
Eggplant Tn Basket
"Put the eggplant in the hasket"
VM: 12
VM:
position
Pick Coke Can
"Pick up the coke can"
URD
VA: 33
VA: basc. backgroundsx lighting texturesx2, camera viewsx2, distractorsx2, can positionx3.
"Move the (object) near the [reference]]
VM:
VM: URDFx4.
Fractal
Move Near
VA: 10
VA: base. distractors, hackgroundsx2, lighting textursxx2, camera viewsx2.
VM: 216
VM: (open/closed) (top. middle, bottom); URDFx4 x4; initial posex9.
Open/Close Drawer
"Open/close the (level] drawer"
VA: 42
VA: base. backgroundsx2. lightingx2, cubinet stylesx2. envx6.
"Put (object) into the (level) drawer"
VM:
VM: URDFx4: initial pose base, backgroundsx2 2,, lighting 2, cabinet stylesx
Put In Drawer
VA: 7
K.1 REAL-WORLD TASKS
Tab. 18 shows the 12 tasks used in real-world evaluation, divided into General and Long-horizon Temporal suites.
General Tasks. Insert Circle: insert circle onto vertical pillar, requiring accurate positioning and insertion. Egg in Pan: place an egg into a shallow frying pan, testing grasp stability and gentle placement. Egg in Oven: put an egg into small oven container, involving more constrained place- ment than the pan. Stack Cups: stack one plastic cup on top of another, evaluating vertical alignment and balance. Stack Blocks: stack a yellow block on top of a red block, focusing on precise spatial alignment. Pick Diverse Fruits: pick a specified fruit from a tabletop with more than ten different fruit types and place it into a basket, testing semantic understanding, visual diversity, and instruction following.
Long-horizon Temporal Tasks. Seq. Push Buttons: push three buttons specified color se- quence, stressing ordered memory and resistance to temporal confusion. Change Food: remove a food item from a plate and replace it with another, requiring multi-step sequencing and correct temporal ordering. Guess Where: cover block with container and later uncover it, testing versible actions and consistent tracking over time. Clean Table Count: clear items from the table one by while pressing counter button after each removal, combining manipulation with plicit progress monitoring. Pick Place Order: pick up carrot, banana, and orange in fixed order and place them into basket, enforcing sequence-sensitive planning under temporal dependencies. Clean Restaurant Table: sort table items by category, placing trash into a trash bin and tableware into a storage bin, representing a long- horizon task with semantic reasoning and complex multi-stage sequencing.
K.2 SIMPLERENV TASKS
Tab. 19 summarizes the tasks in the SimplerEnv benchmark, which consists of two suites; Bridge and Fractal.
29

---

Published as a conference paper at ICLR 2026
Table 20: LIBERO tasks details. We list the language instruction templates and the total number of tasks per suite.
Suite
Language Instruction Templates
# #Tasks
Spatial
pick up the OBJ SPATIAL.REI and place it on the TARGET
10
Object
pick up the FOOD and place it in the CONTAINER
10
open/close the CONTAINER open the DRAWER and put the OBJ inside
Goal
put the OBJ on/in the TARGET push the OBJ to the POSITION of the TARGET turn on the APPLIANCE
10
put both OBJI and OBJ2 in the CONTAINER turn on the APPLIANCE and put the OBJ it and close it
Long-10
put the OBJ in the CONTAINER/APPLIAN
10
place OBJI TARGETI and OBJ2 TARGET2/at REL of TARGET2
pick the OBJ and place it in the caddy COMPARTMENT
open/close CONTAINER/APPLIANC. [and put OBJ on/in it; optionally
sequence with another open/close] open CONTAINER and put OBJ in it put/place OBJ in/on/under TARGET or at REL_POS stack OBJI OBJ2 [optionally place them in CONTAINER] pick up OBJ and put it CONTAINER (basket/tray)
Long-90
90
place MUG on left/right PLATE BOOK in caddy/on/under shelf
turn on/off APPLIANCE (optionally put OBJ on it]
The Bridge suite contains four tabletop manipulation tasks WidowX robot: Spoon Towel, Carrot on Plate, Stack Cube, and Eggplant in Basket. Each task is paired with a single language template, focusing on object placement and stacking primitives.
The Fractal suite builds on RT-1 data with Google robot and defines four tasks: Pick Coke Can, Move Near, Open/Close Drawer, and Put in Drawer. Each task is evaluated under two protocols. Visual Matching (VM) mirrors the real-world setup by varying object positions and URDFs, ensur- ing alignment between simulation and deployment. Visual Aggregation (VA) introduces substantial visual perturbations, including changes in backgrounds, textures, lighting, distractors, and camera views, to stress robustness and generalization. Together, VM and VA yield 336 variants, pro- ducing 2.352 evaluation trials.
K.3 LIBERO TASKS
Tab. 20 outlines the five suites of the LIBERO benchmark: Spatial, Object, Goal, Long-10, and Long-90. LIBERO-Spatial consists of tasks where the same object must be placed across varying target positions. LIBERO-Object focuses on handling diverse objects within a fixed scene layout. LIBERO-Goal contains heterogeneous operations such as opening containers, placing objects, or turning on appliances. performed in an unchanged environment. Long- 10 introduces ten extended tasks that require multiple sub- aub-goals across different scenes, while Long-90 expands this setting to ninety tasks, providing a substantially more challenging benchmark. In total, LIBERO offers 130 tasks in simulation with a Franka robot.
L VISUALIZATION OF MEMORY DEPENDENCE IN REAL-WORLD TASKS
To clarify why real-world temporal tasks require memory, Fig. highlights the key moments in several tasks where the correct action depends past information rather than the current observa- tion. Fig. 12 further provides step-by-step example from the Change Food task, showing a case where the next action becomes ambiguous from single frame and only be resolved by recalling
30

---

Published as a conference paper at ICLR 2026
earlier steps. These examples illustrate that the stronger gains real-robot tasks stem from their inherently memory-dependent nature.
(a) Clean Table Count
Miss
pressed
before"
Repeated
Clean the table one by one, and press the button for each item cleaned
(b) Guess Where
Change Food
What food did I just pick & place? Which food should pick now?
Which is the block under?
Place cover over the block, then remove the cover
Move food off the plate, then the other food on it
Figure 11: Memory-dependent aspects of three real-world robotic tasks. The tasks Clean Table & Count, Guess Where, and Change Food all require tracking past events to make correct decisions, highlighting their inherent temporal dependence.
Ready
Pick Up
Take Away
Confusion
Another
Just begun, or almost over?
Take away place it on plate?
Place corn pick com?
Move left or right? What did just pick
Same confision like "Take Away"
Place Another
Over!
Language Prompt
Move food the plate, then put the other food on
Initial States in Training Data
Same confusion like "Pick Up"
Same confusion
"Ready
Figure 12: Step-by-step example of memory deeendedd behavior. A sequence from the real- world Change Food task showing case where the next action becomes ambiguous without recalling earlier steps.
31

---

Published as a conference paper at ICLR 2026
M QUALITATIVE RESULTS
M.1 REAL-WORLD EVALUATION
We present qualitative examples to complement the quantitative evaluation. Fig. and 14 illustrate rollouts on long-horizon temporal tasks in real-world. Fig. 15 shows general manipulation tasks in real-world.
(a) Seq Push Buttons: push the blue, green, and pink buttons in sequence
Change Food: move food plate, then put the other it
bad
(c) Guess Where: place cover over block, remove cover
Figure 13: Qualitative results of Memory real-worid long-horizon temporal tasks (I),
Representative examples include Seq Push Buttons, Change Food, and Guess Where tasks.
32

---

Published as a conference paper at ICLR 2026
(d) Clean Table Count: cleun the one by one, and press button item cleuned
(e) Pick Place Order: pick up carrol, banana, and orange in order and place them in the basket
(1) Clean Restaurant Table: place all trash items into the trash bin and all tableware into the storage bin
Figure 14: Qualitative results VLA on real-world long-horizon temporal tasks (II). Representative examples include Clean Table & Count, Pick Place Order, and Clean Restaurant Table tasks.
M.2 SIMULATION EVALUATION
Results on simulated environments are visualized in Fig. 16 and 17, covering both Bridge and Fractal suites. Finally, Fig. 18 provides representative trajectories on LIBERO, spanning all five suites.
33

---

Published as a conference paper at ICLR 2026
(a) Insert Circle: insert the circle on the square
(b) Egg Pan: put egg in pan
(c) Egg In Oven: put egg oven
(d) Stack Cup: stack the green the other cup
(e) Stack Block: stack the yellow block the red block
Apple
Apple
Apple
Banans
Carrol
Cpili
Grape
(l) Pick Diverse Fruit: pick up the apple and place in the basket
Figure 15: Qualitative results of Memory VLA on real-world general tasks. Representative ex- amples include Insert Circle, Egg in Pan, Egg in Oven, Stack Cups, Stack Blocks, and Pick Diverse Fruits tasks.
34

---

Published as a conference paper at ICLR 2026
(a) Spoon On Tower: put the spoon on the tower
(b) Carrot On Plate: put the the plate
(c) Stack green cube on yellow
Eggplant Basket: put eggplant the baskel
Qualitative results of Memory VLA on SinplerEn tasks. Representative examples include Spoon on Tower, Carrot on Plate, Stack Cube, and Eggplant in Basket tasks.
Figure 16:
(a) Coke Can pick up the coke can
(b) Move Near: the orange the Pepsi
(c) Open Close Drawer: open
close the top drawer
(d)Put Drawer: put apple into the top drawer
Figure 17: Qualitative results of MemoryVLA on SimplerEnv-Fracha tasks. Representative examples include Pick Coke Can, Move Near Open/Close Drawer, and Put in Drawer tasks.
35

---

Published as a conference paper at ICLR 2026
(a) Spatial: pick up black bowl wooden place plate
(b) Object: pick up the tomato sauce and place in the basket
(c) Goal: open lop drawer put bowl inside
(d) Long: turn on the stove and put the moka pot on it
(c) LIBERO-90. close the microwave
Qualitative results of Memory LIBERO tasks. Representative examples in- clude tasks from Spatial, Object, Goal, Long-10, 10, and Long-90 suites.
Figure 18:
36


Reviews:
[{"review_id": "cMhA83ROXP", "reviewer": "Reviewer_F2kP", "summary": "The paper tackles non-Markovian, long-horizon robotic manipulation where single-frame VLAs fail. It proposes MemoryVLA, a Cognition-Memory–Action framework that (i) converts a single RGB frame + instruction into perceptual tokens (DINOv2+SigLIP compressed via SE-bottleneck) and a single cognitive token (EOS from LLaMA-7B), (ii) stores both streams in a Perceptual–Cognitive Memory Bank (PCMB), and (iii) performs retrieval (cross-attention + timestep PE), gate fusion, and consolidation (adjacent-pair merge by cosine similarity) before a DiT+DDIM action head predicts a 16-step continuous 7-DoF trajectory. The authors conduct an extensive evaluation across 150+ tasks in simulation (SimplerEnv, LIBERO) and the real world.", "questions": "* Although the model shows strong robustness to many OOD variations, the performance drops sharply when viewpoint changes in Appendix (Sec. C). Does that suggest the learned perceptual features are highly view-dependent and that the memory module may be memorizing visual details rather than an abstracted spatial representation?\n* How does the memory length affect the performance of the model?\n* How does different memory modules work independently? Can you provide a qualitative example of a task failure that occurs with only cognitive memory (i.e., lacking perceptual detail) and a failure that occurs with only perceptual memory (i.e., lacking cognitive gist)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* Clear structure with good motivation: the task of handling of non-Markovian tasks is interesting and significant in robotics, where the motivation fused memory term within the architecture design.\n* Extensive evaluation: the authors evaluate MemoryVLA across three different robots, three distinct simulation benchmarks (SimplerEnv-Bridge, SimplerEnv-Fractal, LIBERO), and a set of 12 real-world tasks. This comprehensive evaluation on 150+ tasks with 500+ variations provides high confidence in the method's effectiveness and generalizability.\n* Significant performance gain: the model achieves state-of-the-art performance across all benchmarks. The standout result is the +26 point improvement over the next-best baseline on real-world long-horizon tasks.", "weaknesses": "* The ambiguity of optimal memory length: from the ablation study in Table 5, it suggests that a memory length of $L=16$ is optimal (71.9% success), while the performance worsens at $L=64$ (67.7%). However, in the Appendix, the authors state that a memory length of $L=256$ was used for real-world long-horizon tasks. There lacks of in-depth analysis of how memory length is associated with the actual performance.\n* The mechanism of using single cognitive token: for complex tasks require multiple latent hypotheses, one EOS token could be lossy. The paper doesn’t probe whether more cognitive capacity helps or hurts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CFXg9Zfcf0", "reviewer": "Reviewer_PZt6", "summary": "The paper introduces a new model, MemoryVLA, which employs a specialized memory bank designed to better handle temporal dependencies.", "questions": "1. The main question is how the proposed model performs on tasks from Mikasa-Robo [1] and MemoryBench [2].\n2. In [1], it was shown that using an action chunk of large size can circumvent the need for a memory mechanism to solve tasks. In MemoryVLA, a chunk of size T = 16 is used, predicting 16 steps ahead. How would the model behave if this value were reduced? In which cases does performance improvement come from using a long chunk, and in which cases from the memory mechanism itself?\n3. What is the number of steps (actions) required to solve the tasks (mean, minimum, maximum, and median), including on the real robot?\n4. How do the elements retrieved from the memory bank correspond to the task being solved at a given moment?\n5. Why does the proposed model perform so much better than CogACT in real-robot experiments compared to simulation? Were the models trained under comparable conditions?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "1. Incorporating memory mechanisms into VLAs is a highly relevant and important research direction.\n2. The paper presents a large number of experiments, including those conducted on a real robot.\n3. The work is well written and easy to read.", "weaknesses": "1. Despite the large number of experiments, the main drawback of the paper is that most of the tasks used do not actually require a memory mechanism.  The authors should conduct comparisons on specialized robotics benchmarks focused on memory-based tasks, such as Mikasa-Robo [1] and MemoryBench [2]. Without these experiments, it is impossible to properly evaluate the effectiveness of the proposed memory mechanism.\n2. The results on LIBERO outperform Discrete Diffusion VLA [3] by only 0.3, even though the latter does not use any memory mechanisms. This again raises questions about the suitability of the chosen benchmarks for evaluation.\n3. The functioning of the memory mechanism is not demonstrated clearly, it can only be inferred indirectly from the overall model performance. It is important to show that the memory bank retrieves relevant elements, providing direct evidence of how the mechanism contributes to task solving.\n\nIn its current state, I believe that, despite addressing an important direction, the paper does not sufficiently demonstrate that the proposed memory mechanism truly helps solve complex memory-dependent tasks. This is primarily due to testing mostly on simple tasks that do not require memory, as well as the lack of an in-depth analysis of the memory mechanism itself.\n\nI am willing to reconsider my evaluation if these shortcomings are addressed.\n\nReferences:\n1. Cherepanov, Egor, et al. \"Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning.\" arXiv preprint arXiv:2502.10550 (2025). \n2. Fang, Haoquan, et al. \"Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation.\" arXiv preprint arXiv:2501.18564 (2025).\n3. Liang, Zhixuan, et al. \"Discrete diffusion vla: Bringing discrete diffusion to action decoding in vision-language-action policies.\" arXiv preprint arXiv:2508.20072 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EuQj2ed0jx", "reviewer": "Reviewer_5ryq", "summary": "The paper tackles core gap in VLA which is weak temporal reasoning for long-horizon, non-markovian manipulation, and they do so by drawing on cognition-memory-action architecture with two complementary memory systems. A pretrained VLM converts observations into (i) perceptual tokens and (ii) higher-level cognitive tokens that together serve as working memory for immediate control. In parallel, a Perceptual-Cognitive Memory Bank accumulates both low-level details and high-level semantic “gist.” At each step, working memory retrieves decision-relevant entries from the bank, adapts/fuses them with current tokens, and updates the bank by merging redundancies. A memory-conditioned diffusion action expert then produces temporally aware action sequences.", "questions": "The paper should address its two major weaknesses; without doing so, its claims will remain insufficiently supported. I would consider change the rating if my questions can be addressed.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The working-memory vs. long-term (episodic + semantic) split is directly inspired by human memory and mapped cleanly to a VLA stack (perceptual/cognitive tokens + Perceptual-Cognitive Memory Bank). This makes the temporal modeling choice easy to justify and reason about.\n\n2. Converting observations into perceptual and cogntiive tokens enable lightweight retrieval, fusion and consolidations.\n\n3. Good performance on SimplerEnv-Bridge benchmark and LIBERO.", "weaknesses": "1,Benchmark mismatch (memory not actually required).\n\nFundamentally, the simulation benchmark used does not evaluate memory: the tasks appear in-distribution, short-horizon, and solvable without non-Markovian reasoning. I recommend evaluating on a benchmark that explicitly requires memory, such as Memory-Bench (from SAM2Act), to substantiate the paper’s claims.\n\n2.Inadequate baselines (no memory or long-context retrieval).\n\nThe chosen baselines are not memory-enhanced and do not leverage long context or retrieval, making it difficult to attribute gains to the proposed method. Please compare against baselines that incorporate memory or retrieval to fairly assess effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JYoGamsT4w", "reviewer": "Reviewer_MmT3", "summary": "This paper introduces MemoryVLA, a Cognition-Memory-Action framework for robotic manipulation that addresses the limitation of mainstream VLA models in handling long-horizon, temporally dependent tasks. Inspired by human memory systems, the approach features a Perceptual-Cognitive Memory Bank that consolidates low-level visual details and high-level semantic information, which working memory retrieves and fuses with current observations to condition a diffusion-based action expert.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper identifies a concrete limitation in existing VLA models (ignoring temporal context) and proposes a practical memory bank mechanism with retrieval, fusion, and consolidation operations. The dual-stream design (perceptual + cognitive tokens) with gated fusion is technically sound and the memory consolidation strategy effectively manages computational costs.\n\n2. The method achieves substantial improvements over strong baselines, with comprehensive experiments across 3 robots, and thorough ablations validating each design choice. The real-world deployment on both Franka and WidowX robots demonstrates practical applicability beyond simulation.", "weaknesses": "1. The paper does not report inference time and memory footprint comparison with baselines. The memory retrieval via cross-attention at each timestep, especially with memory lengths up to 256, likely incurs significant computational overhead that could limit real-time deployment.\n\n2. The paper lacks visualization or analysis of retrieved memory contents. It's unclear whether the memory bank actually retrieves semantically/temporally relevant contexts or if the gains simply come from having more visual history. Attention weight visualization or case studies showing retrieved frames would strengthen the claims.\n\n3. Ablations (Tables 5-6) are only conducted on SimplerEnv-Bridge. It's unclear if the design choices (e.g., memory length=16, gate fusion, token merge) generalize to other benchmarks like LIBERO or real-world tasks where temporal dependencies may differ.\n\n4. Limited generalization analysis:\n* Task generalization: While OOD robustness is tested with visual variations (backgrounds, lighting, occlusion), there's no evaluation of zero-shot generalization to unseen task categories, which is crucial for a general-purpose VLA model.\n\n* Memory capacity analysis: For long-horizon tasks, is memory length of 256 sufficient? The paper doesn't analyze what happens when task horizon exceeds memory capacity, or whether the consolidation strategy causes information loss for very long sequences.\n\n* Benchmark selection for ablations: Tables 5-6 conduct ablations only on SimplerEnv-Bridge, which arguably doesn't require strong temporal dependencies (as the tasks are relatively simple pick-and-place). Ablations should be conducted on benchmarks where temporal reasoning is critical (e.g., real-world long-horizon tasks) to validate design choices.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)