HAMLET: Switch Your Vision-Language-Action Model into a History-Aware Policy
Authors: Myungkyu Koo
Daewon Choi
Taeyoung Kim
Kyungmin Lee
Changyeon Kim
Younggyo Seo
Jinwoo Shin
Venue: ICLR 2026 Poster (2026)
Keywords: Vision-language-action models
Robot manipulation
Primary area: applications to robotics, autonomy, planning
TL;DR: HAMLET is a scalable framework that enables pre-trained VLAs to leverage historical information without costly retraining from scratch.

Abstract: Inherently, robotic manipulation tasks are history-dependent: leveraging past context could be beneficial. However, most existing Vision-Language-Action models (VLAs) have been designed without considering this aspect, i.e., they rely solely on the current observation, ignoring preceding context. In this paper, we propose HAMLET, a scalable framework to adapt VLAs to attend to the historical context during action prediction. Specifically, we introduce moment tokens that compactly encode perceptual information at each timestep. Their representations are initialized with time-contrastive learning, allowing them to better capture temporally distinctive aspects. Next, we employ a lightweight memory module that integrates the moment tokens across past timesteps into memory features, which are then leveraged for action prediction. Through empirical evaluation, we show that HAMLET successfully transforms a state-of-the-art VLA into a history-aware policy, especially demonstrating significant improvements on long-horizon tasks that require historical context. In particular, on top of GR00T N1.5, HAMLET achieves an average success rate of 76.4% on history-dependent real-world tasks, surpassing the baseline performance by 47.2%. Furthermore, HAMLET pushes prior art performance from 64.1% to 66.4% on RoboCasa Kitchen (100-demo setup) and from 95.6% to 97.6% on LIBERO, highlighting its effectiveness even under generic
robot-manipulation benchmarks.

Published as a conference paper at ICLR 2026
HAMLET: SWITCH YOUR VISION-LANGUAGE ACTION MODEL INTO A HISTORY-AWARE POLICY
Myungkyu Koo"!
Daewon Choi
Taeyoung Kim' Jinwoo S,
Kyungmin Leel
Changyeon Kim
Younggyo Seot2
KAIST UCC Berkeley 'RWWRDD
(Jamesko00503, daeona0920jekaist..c
ABSTRACT
Inherently, robotic manipulation tasks history-dependent: leveraging past con- text could be heneficial. However, most existing Vision-Language-Actte Action models (VLAs) have been designed without considering this aspect, i.e., they rely solely on the current observation, ignoring preceding context. In this paper, we propose HAMLET, scalable framework to adapt VLAs to attend to the historical context during action prediction. Specifically, we introduce moment tokens that compactly encode perceptual information at each timestep. Their representations are initialized with time-contrastive learning, allowing them to better capture temporally distinc- tive aspects. Next, we employ lightweight memory module that integrates the mo- ment tokens across past timesteps into memory features, which then leveraged for action prediction. Through empirical evaluation, we show that HAMLET suc- cessfully transforms tfate-of-the-al VLA into history demonstrating significant improvements on long- horizon tasks that require histori- cal context. In particular, on top of GROOT N1.5, HAMLET achieves an average success rate of 76.4% on history-dependent line performance by 47.2%. Furthermore, HAMLET pushes prior art performance from 64.1% 1% to 66.4% on RoboCasa Kitchen (100-demo setup) and from 95.6% 97.6% on LIBERO, highlighting its effectiveness even under generic robot- manipulation benchmarks. Project page: https://myungkyukoo, htw.:/ggyyyooooguuoooooaaaaa .ithuub.ooo/aalee//////
policy, especially
world tasks, surpassing the base-
INTRODUCTION
Dision-Language-Attin Action models (VLAs; Zitkovich al. 2023; Kim al. 2024: Black et al. 2025; Pertsch et al. 2025; Li et al. 2024a; Qu et al. 2025; Bjorck et al. 2025b) have shown their promise in robotic policy learning by leveraging large-scale pre-trained Vision-Language Models (VLMs; Beyer et al. 2024; Chen et al. 2023; Driess et al. 2023; Karamcheti et al. 2024; Touvron et al. 2023) with diverse robot-specific datasets (Walke et al., 2023; O'Neill et al., 2024; Khazatsky et al., 2024). They typically adopt single- frame assumption. predicting each action solely from the current observation. However, such reliance on a current snapshot fundamentally limits their capability, since robotic manipulation tasks are intrinsically distory-dependent. For instance, consider a simple scenario of placing an object on a table. The decision to move the arm depends on prior context-specinfee context--peeiffiallyy whether the object has already been grasped. When restricted to the current frame, the policy may struggle to determine the proper next action, particularly if the object is occluded.
Despite being a desirable property, incorporating history-awareness during pre-training is viewed as a costly design choice. A major challenge is that leveraging historical context incurs substantial computational overhead. For example, we observe that naively appending only four additional past observation frames to the VLA input slows down the forward pass by ~35% and increases peak memory consumption by ~3.6x (see Multi-frame in Table 4). In particular, the inflated memory footprint further restricts scalability by reducing feasible batch sizes compared to the single-frane frame setting. Together, these observations raise a key research question: How can we integrate history- awareness into pre-trained VLAs without resorting to costly pre-training from scratch?
"Equal contribution.
I Equal advising.

---

Published as a conference paper at ICLR 2026
(a) History-dependent Long-horizon Tasks
(b) Success Rates
"Cover the cube with the nearest cup, then stack the other top of it."
Cover and Stack)
GROUT N1.5
5%
62.5%
Multi-frme
70.8%
HAMLET (oura)
75 2%
66
The cube under the blue cup...
Wait, where is the cube?
Partel
"Swap the two cubes, starting with the blue one."
Swap Cubes]
]
GROOT N1.5
5%
37.5%
Multi-frame
99 5%
91.7%
HAMLET (ours)
55.3% 95
Partbal
The blue cube was on the right...
Wait, which cube to grasp next?
Figure I: Examples of history-dependen long-horizon tasks. (a) Real-world tasks that involve
cases such as object occlusion (upper) or multi-step reasoning (lower) are inherently non-Markovian: proper actions cannot be determined from the current observation alone. (b) Success rates on these tasks show that HAMLET significantly outperforms both GROOT N1.5 and the naive baseline, demonstrating its ability to leverage historical context for reliable long- nog-horizon control.
To tackle this problem, we propose HAMLET, a fine-tuning framework for VLAs that introduces History-Awan Memory with LEarned Tokens. Our framework consists of two components: (a) moment tokens, which summarize the instantaneous VLM representations at each timestep. and (b) memory module, which consolidates moment tokens across different timesteps to produce temporally informed condition for action prediction. The moment tokens are appended to the VLM input at each timestep and initialized with time-contrastive learning (Sermanet et al., 2018), which encourages distinctiveness across timesteps. This initialization enables the moment tokens emphasize task- relevant dynamics while suppressing redundant information such as static backgrounds (see Figure 4 for details). Building this, we incorporate lightweight memory module that stores and integrates moment token representations across timesteps. This design is motivated by the observation that not all moments are equally informative; treating every timestep with equal importance can introduce redundancy and obscure critical cues (see Moment Concat. in Table 5c).
To validate the effectiveness and generality of HAMLET, we conduct comprehensive experiments across both real-world and simulation environments. We first evaluate HAMLET on the long-horizon, real-world tasks that require reasoning over past trajectories. We show that HAMLET improves performance by 47.2% over the naively fine-tuned VLA, which demonstrates the effectiveness of exploiting historical information for real-world robot policy learning. We further examine the generality and applicability of HAMLET across different VLA backbones. When fine-tuning GROOT N1.5 (Bjorck et al., 2025a) on the RoboCasa (Nasiriany et al., 2024) Kitchen dataset, HAMLET achieves an average success rate of 66.4%, compared to 64. 14. for the baseline. Similarly, when (Li et al., 2024c) dataset, HAMLET
applied to CogACT (Li et al., 2024a) on the SimplerEnv-Bride attains 63. 63.5%, substantially improving over the baseline performance of 52.1% These results highlight that incorporating history- consistently yields benefits across diverse VLA policies, and that HAMLET provides consistent improvements flexible, plug-in manner.
Contributions. Our contributions are as follows:
Motivated by VLAs' reliance on the current observation alone, we propose HAMLET, an easily integrable framework that enables history- in pre-trained VLAs.
We introduce moment tokens, initialized with time-contrastive learning, to capture key temporal cues at each timestep. Building this, we design a lightweight memory module that selectively aggregates these tokens across timesteps to produce history-aware features for action prediction.
We validate HAMLET across both real-world and simulation benchmarks, achieving substantial gains over state-of-thar oftte-of-the-ar the-art baselines. By alleviating backbone models' reliance on the current observation, HAMLET delivers consistent improvements, especially with the strongest benefits
2

---

Published as a conference paper at ICLR 2026
on long- long-horizon tasks. Importantly, its backbone-agnostic design allows seamless and efficient integration into diverse VLAs without requiring any additional large-scale pre-training.
2 RELATED WORKS
Vision-Language-AActie models (VLAs). Since collecting high-quality and large-scale robot datasets is challenging, traditional robot learning (Shafiullah et al., 2022; Cui et al., 2022; Chi et al., 2023; Lee et al., 2024) has relied on task-specific data, which cover only a narrow distribution of environments, objects, and task instructions. To address this limitation, recent studies (Zitkovich et al., 2023; Driess et al., 2023; Kim et al., 2024; Pertsch et al., 2025; Bjorck et al., 2025a; Li et al., 2024a; Black et al., 2025) propose building generalist robot policies by utilizing internet-scale VLM priors to low-level action prediction, thereby transferring semantic understanding to robotic control. Early VLAs (Zitkovich al., 2023; Kim al., 2024; Driess al., 2023) discretize the continuous action space and directly predict actions as tokens, demonstrating that pre-trained VLMs can be adapted to robot control. More recent approaches (Black al., 2025: Bjorck et al., 2025b; Li et al., 2024a) leverage VLM representations to condition action experts on diffusion or flow- now-matching, enabling more accurate action prediction. For example, Black al. (2025) conditions a flw-matching head on VLM features to produce action chunks per step, and Li et al. (2024a) systematically compares action modules, finding that diffusion action transformers scale favorably when conditioned on VLM representations. However, these models typically generate actions based only on the current observation, limiting their capability to accurately recognize the current state and determine precise actions. In this work, focus on recent pre-trainee pre-trained VLAs with diffusion iistusiioo-bbseddd action heads, and demonstrate how our framework can enhance their performance by incorporating historical context.
Memory architectures. Traditionally, mag-horizon tasks have been framed non-Markovian problems, requiring policies to integrate memory to leverage past observations and actions. In reinforcement learning, recurrent policies (Hausknecht & Stone, 2015) and later Transformer-baser variants (Parisotto et al., 2020) introduced memory mechanisms that improved performance on partially observable and long-horizon benchmarks. In natural language processing, explicit memory architectures have been widely explored, ranging from end-to-ent tond memory networks (Sukhbaatar al., 2015) to retrieval- based approaches (Khandelwal al., 2019; Lewis et al., 2020), all designed to enhance long-context reasoning and knowledge integration. These advances highlight the importance of dedicated memory modules for tasks requiring extended temporal or contextual awareness. In robotics, prior works (Wen et al., 2020; Seo et al., 2023; Li et al., 2024b; Huang et al., 2025) have incorporated past observations into policy learning. but most approaches (Wen et al., 2020; Seo et al., 2023) rely simple policy architectures (e.g., MLPs or RNNs) with limited generalizability and require training from scratch (Li et al., 2024b; Huang et al., 2025), leading to heavy computational cost. Consequently, memory design for large-scale pre-trained VLAs remains under-explored. As concurrent effort, Shi et al. (2025) proposes architectures inspired by human memory systems, trained from scratch, and demonstrates promising improvements temporally dependent tasks. Distinet from this line of work, our approach augments pre-trained VLAs with few learnable tokens and a without retraining.
lightweight memory module, allowing them attend to
METHOD
In this section, introduce HAMLET, a plug-in framework that adapts pre-trained Vision-Language- Action models (VLAs) to attend the historical context. Formally, let o = [I..... IF-) be the sequence of visual observations timestep f, and e be task instruction. The VLA Fo processes these inputs through its Vision-Language Model (VLM) backbone to obtain hidden representation h;:
h Fo(o,, c).
Then, the representation h, is used as a condition for the action expert Ay to predict a sequence of future actions, namely action chunking (Zhao et al., 2023; Chi al., 2023):
[as, a4,,.,,... - Ay (h,,
where s; denotes the robot's proprioceptive state at timestep t. After executing the predicted action sequence, the environment returns a new observation 0,+ which serves as the next input to the VLA. Here, goal is to augment h, with informative representations from previous timesteps
3

---

Published as a conference paper at ICLR 2026
Original path
Concat:
Appended path for history-awareness
State
Cross Attention
Self Alt--Attttiion
Cross- Cross-Atteentie PetssAAttentionn
Self Atte-ttno
Noised
Memory Module
Action
Action
Action Expert (DIT)
VLM
Memory Feature
Text Tokenizer
Vision Encoder
Vision Encoder
Text Tokenizer
Moment Tokens
Memory Module
"You are helpful
"Cover the cube with the nearest cup."
Cached Past Tokens
Current Tokens
assistant."
Building on pre-trained VLA, HAMLET adds two key components: moment tokens, appended to the VLM input and initialized with time-contrastive learning to capture task-relevant representations at each timestep, and a lightweight memory module that aggregates these tokens across timesteps for history-aware action prediction.
Figure 2: An overview of HAMLET.
(i.., Or-k, OL-2k....., to enable the action expert to effectively utilize long-horizon context. To achieve this goal, propose complementary components: (i) moment tokens, which compress the information at each timestep into a compact representation (see Section 3) and (ii) memory
module, which aggregates moment tokens across timesteps to yield a temporally-enricher condition for action prediction (see Section 3.2). The overall framework is illustrated in Figure
3.1 CONTEXT COMPRESSION VIA MOMENT TOKENS
To leverage historical context, we first consider how to effectively store information from each timestep. Retaining the raw visual observation latency and memory costs (see Table 4 in Section 4.3) and often contains redundant or static elements that might provide irrelevant signal (Xu et al., 2025; Yang et al., 2025). To address this, propose to compress the observation at each timestep into a concise representation that preserves task-relevant information while filtering redundancy. This motivates the design of moment tokens.
at every timestep is suboptimal: it incurs high
Use of moment tokens. At each timestep f, we append set of learnable moment tokens m, E IR" the input sequence of the VLM, where n.n the number of tokens and the embedding dimension. Given visual observation and task instruction we append moment token m; to them and feed the combined input to the VLM encoder Fo to produce a hidden representation:
[h, m1] Fo( c; mz)),
where [. denotes concatenation of token sequences. Along with the hidden states h, we extract the representation m of moment tokens, which act as compact, context-aware summaries of the scene at timestep t. Due to the causal attention operator in the VLM, moment tokens attend to the current visual observation o, and the task instruction c. As result, m serves as a compressed representation of each timestep, which subsequently be stored and aggregated by the memory module.
Time-contrastive learning. To encourage moment tokens to encode temporally discriminative cues at each timestep. draw inspiration from contrrtttttttvee network (Sermanet et al., 2018; Nair et al., 2022; Ma et al., 2023), while adapting the design of positive pairs to image-augmented samples with photometric, blur, noise, and occlusion perturbations. For trajectory [On ... Or-11 and task instruction c, we extract moment token representations m at each timestep using Eq. (3).
To construct the contrastive objective, for each timestep observation o,, We then generate a positive z from an augmented view of the same observation and a hard negative z; from a different timestep A t within the same trajectory. Formally, let 2, (mm) denote the projected moment-token representation produced by projection head 9-)).
form anchor from the current
4

---

Published as a conference paper at ICLR 2026
We then optimize the following time-contrastive learning objective:
B
exp(sim(zz.. 2, //T)
LrCL. L 27
log ==1 exp (sim(z,, (z, z," //T) + exp(sim(zz) z, )/7)*
(4)
where sam b) denotes cosine similarity and T is temperature hyperparameter. The summation indexes the anchors in the minibatch. This initialization encourages the moment tokens to align with representations from the same timestep while remaining discriminative across timesteps, enabling m, to capture unique, timestep-spest speccfcc cues while suppressing static components. During this stage, we freeze the VLM Fo to ensure that the loss does not distort its pre-trained representation.
3.2 MEMORY CONSOLIDATION VIA MEMORY MODULE
We now present a memory module to integrate the moment token representations m across timesteps for action prediction. We observe that simply concatenating these representations does not directly improve performance (see Moment Concal. in Table 5c). Hence, we employ lightweight Transformer Mr that selectively attends to informative past timesteps while ignoring less relevant ones.
Design of memory module. To incorporate historical context beyond current timestep, we introduce a memory module M& that aggregates moment token representations across timesteps. Specifically, we employ shallow Transformer (Vaswani et al., 2017) that attends over past moment tokens via causal self-attention. We form a history matrix by stacking the most recent moment tokens:
M' [ ...; mi m/] E 1xd
(5)
where k the action-chunk length from Eq. (2), the history length, and L= T number of tokens. From M', the memory module applies standard self-attention:
7m is the total
Q = M'',, K M'W
M'W.,
softmax
QK
-CVV
(6)
where C is a causal mask ensuring the proper encoding for sequential trajectory. H is mapped through the Transformer's output projection, producing M' &Rdd Then, we take the last n'm rows of M', denoted m as the castery-augmented moment token representation for timestep
Integration into action prediction. The original VLM representation h, and fed into the action expert A& to predict the next & actions.
feature mn is concatenated with the
[a a++1, .... a++4-1] A& ([,; m'),.
The overall training procedure follows that of standard VLA models, where the pipeline is trained end-to-end with the action prediction loss (Bjorck et al., 2025b; Li et al., 2024a; Black al., 2025).
EXPERIMENTS
We design experiments investigate the following questions:
Does applying HAMLET to existing VLAs enhance performance on long-horizon real-world tasks that require reasoning over past trajectories? (Table I in Section 4.2)
HAMLET also beneficial on generic mobot-manipulation benchmarks? (Table 2, 3 in Section 4.2)
Can HAMLET be seamlessly applied across different pre-trained VI.As? (Table 3 in Section 4.2)
How does HAMLET perform in terms of computational overhead, effective design choices, and transferability to unseen datasets? (Table 4, 5, 6 in Section 4.3, respectively)
4.1 EXPERIMENTAL SETUPS
Datasets. We evaluate HAMLET real-world tasks that require reasoning past trajectories, as well as on diverse simulation benchmarks (Figure 5). In the real-world environment, we design three handcrafted tabletop tasks: (i) Pick-and-Place Twice, where the robot moves a cube between two sides twice; (ii) Cover-and-Staud where the robot covers a cube with one cup and then stacks it with another; and (iii) Swap Cubes, where the robot swaps the positions of two cubes using auxiliary
5

---

Published as a conference paper at ICLR 2026
(a) Tick-and-Place Twice
Should release, but carries away!
GRODT N1.5
Properly released.
HAMLET
(b) Cover-and-Stack
Approaches the same again!
GROOT N1.5
Properly reaches the opposite.
HAMLE LET
Figure 3: Example rollouts of real-world tasks. We present example rollouts executed by our HAMLET and GROOT N1.5, respectively. While HAMLET predicts proper next actions, (a) GROOT N1.5 is confused about whether it should lift release the cube, and in (b) it fails to identify which cup has a cube underneath, due the absence historical context.
Table I: Real-world evaluation results. We report the success rate (%, over 24 trials per task) on three real-world tasks: partial success rates for columns (PnP Once, Cover Cube, Stage Cube), and 'Success' for full completion. Bold and underline indicate the best and runner-up results, respectively.
Pick-and-Plaan Place Twice
Cover-and-Stane Stack
Swap Cubes
Method
History?
PnP Once
Success
Cover Cube
Success
Stage Cube
Success
Avg.
To
54.2
25.0
83.3
12.5
31.9
TAST
54.2
66.7
GROOT N1
54.2
25.0
79.2
33.3
75.0
33.3
30.6
GROOT N1.5
54.2
12.5
62.5
37.5
87.5
37.5
29.2
Multi-frame
91.7
HAMLET (Ours)
91.7
95.8
79.2
95.8
76.4
site. For the simulation environment, conduct experiments three widely-used benchmarks: RoboCasa (Nasiriany et al., 2024) Kitchen, LIBERO (Liu et al., 2023) and SimplerEnv--rrin (Li al., 2024c), which consist of multi-step household manipulation tasks spanning diverse objects and configurations. Further details including real-words robot setups are provided in Section A.2.
Baselines. We design baseline comparisons according the target benchmark. We primarily evaluate on GROOT N1.5 5 (Bjorck et al., 2025b) and further assess generalization CogACT (Li et al., 2024a). For real-world tasks and simulation benchmarks (RoboCasa Kitchen and LIBERO), we compare HAMLET on GROOT NI 1.5 with representative baselines: (Black et al., 2025), -FST (Pertsch we evaluate HAMLET on CogACT against the reported performances of Open (Kim et al., 2024), Octo (Team et al., 2024), RoboVLM (Liu et al., 2025) and SpatialVLA (Qu et al., 2025). For comparison with methods that utilize historical context, implement the multi-frame baseline, which stores past observation frames and concatenates them into the VLA input (see Section A.3 for details).
al., 2025), and GROOT N1 (Bjorck et al., 2025b). SimplerEnv-Bride
Implementation details. We apply HAMLET to each VLA following the original fine-tuning setup of that model, without heuristic hyperparameter tuning. Instead, we adopt the training configurations specified for each backbone (e.g., learning rate, optimizer, and freezing of the VLM backbone). By

---

Published as a conference paper at ICLR 2026
Table 2: Simulation benchmark results on GROOT N1.5. We compare HAMLET with baseline methods on RoboCasa Kitchen and LIBERO. For RoboCasa Kitchen, we report the average success rate (%) across 24 tasks with models trained using 30, 100, or 300 demonstrations per task. For LIBERO, each metric is the average success rate (%) across 10 tasks per suite, with training performed joindy on all suites. All the results are reproduced by us, except for those of GROOT NI on RoboCasa Kitchen. Bold and underline indicate best and runner-up results, respectively,
RobuCasa Kitchen (# of demos)
LIBERO (lask suite)
Method
30
100
300
Spatial
Object
Goal
Long
Avg.
TO
47.8
58.7
62.5
97.2
97.2
93.6
89.2
94.3
Ro-FAST
29.8
60.2
63.6
96.0
96.4
91.6
85.0
92.3
GROOT
17.4
32.1
49.6
95.6
97.6
94.2
89.6
94.3
GROUT N1.5
47.8
62.6
64.1
98.2
99.4
97.2
87.8
95.6
Multi-frame
44.0
59.3
60.8
81.4
97.2
89.4
79.4
86.8
HAMLET (Ours)
52.5
65.4
66.4
99.0
100.0
99.2
92.2
97.6
Table 3: Simulation benchmark results on CogACT. We compare HAMLET with baseline methods on the SimplerEnv-Bridge benchmark. Each metric reports the success rate (%) four Widow X tasks in SimplerEnv, with separate reporting for grasp success and full success. 'Avg.' denotes the average full success rate (%) across the four tasks, and all CogACT results are faithfully reproduced by us. Bold and underline indicate best and runner-up results, respectively.
Spoon on Towel
Carrot on Plate
Stack Block
Eggplant in Basket
Method
Grasp
Success
Grasp
Success
Grasp
Success
Grasp
Success
Avg.
OpenVLL
4.1
0.0
33.3
0.0
12.5
0.0
8.3
4.1
1.0
Octo-Base
34.7
12.5
52.8
8.3
31.9
0.0
66.7
43.1
16.0
Octo-Small
77.8
47.2
27.8
9.7
40.3
4.2
87.5
56.9
30.0
RoboVLM
54.2
29.2
25.0
25.0
45.8
12.5
58.3
58.3
31.3
SpatialVLA
20.8
16.7
29.2
25.0
62.5
29.2
100.0
100.0
42.7
CogACT
87.5
58.3
41.7
37.5
70.8
20.8
91.7
91.7
52.1
Multi-frame
83.3
50.0
79.2
50.0
70.8
20.8
70.8
70.8
47.9
HHMMLTT (Ours)
91.7
75.0
83,3
62.5
75.0
16.7
100.0
100.0
63.5
default, we use moment tokens of length 4, a 2-layer Transformer as the memory module, and history length Full hyperparameters and implementation details are provided in Section A.3.
4.2 MAIN RESULTS
Real-world evaluation. For real-world environment, we train each model for each task independently and evaluate their performance by averaging the success rates over 24 trials per task. For each task, partial success rates are also reported, where the criteria are: (i) Pick-and-Plans Once: pick and place the cube at the correct site once; (ii) Cover Cube: cover the first cube with the nearest cup; and (iii) Stage Cube, stage the cube to the auxiliary site. As shown in Table and Figure 3, the base model (GROOT N1.5) struggles with these tasks, achieving only 12.5% success on Pick-and-Place Twice and often becoming confused about which direction move (see more rollouts in Section B.2). Notably, applying HAMLET to GROOT N1.5 yields substantial improvements across all tasks, achieving an average improvement of 47. 47.2% and underscoring its effectiveness in leveraging historical context.
Simulation benchmarks. To validate generalizability of HAMLET across generic benchmarks, we evaluate on the standard simulation benchmarks, RoboCasa (Nasiriany et al., 2024) and LIBERO (Liu et al., 2023), As shown in Table 2, naively extending GROOT N1.5 with multi-frame inputs degrades the baseline performance by 3.3% in RoboCasa (100 demos) and 8.8% LIBERO. This highlights an inherent weakness of this approach: (a) by conditioning only on consecutive frames during training, the model struggles to generalize to test environments with dynamically varying observations, and (b) merely accessing more past information can cause the policy to pick up spurious temporal correlations, a phenomenon also known as causal confusion (Wen et al., 2020; De Haan et al., 2019; Seo et al., 2023). On the other hands, HAMLET, when applied top of GROOT N1.5, successfully improves performance across benchmarks: in LIBERO, pushes the prior best score 95.6%, near-saturated success rate-up to 97.6%. This supports the advantage of
7

---

Published as a conference paper at ICLR 2026
Attenton weihts
0.1
B:3k
2.2k
6k
(a) Moment token attention over images
(b) Memory attention across timesteps (at timestep t)
Figure 4: What does the memory network memorize? (a) Visualization of self-attention map of moment tokens over input images inside the VI.M, showing that they concentrate strongly on task- relevant regions. (b) Normalized self-attention weights of the memory module across the moment token sequence, indicating which timesteps contribute most to the memory features.
design: since HAMLET still receives dingle-frame inputs via external memory module, it effectively exploits historical context while preserving single-frame VLA's generalizability.
Generalization to other VLAs. We further validate the scalability of our framework in transforming existing VLAs into history-aware policy beyond GROOT N1.5. Specifically, we consider CogACT eLi al., 2024a) as base model, which is another pre-trained VLA based on diffusion policy, on the Simpler-WidowX
Simpler- WidowX simulation benchmark (Walke et al., 2023). Table 3 reports both partial and full success rates across four tasks. Similar other simulation results (e.g.. Table 2), the multi-frame baseline fails to consistently improve success rate across tasks, possibly due to the its pool generalizability. In other hands, HAMLET still demonstrates clear improvements: it not only improve partial success rates across all tasks but, more importantly, significantly improves final task completion, achieving an average full success rate of 63.5%, compared the original 52.1 %. These highlight HAMLET's flexibility as an easily integrable fine-tuning framework, without the need for costly re- re-training.
4.3 MORE ANALYSIS
We further analyze the individual components in HAMLET and its efficiency over baselines. Through- this section, unless otherwise specified, we consider the GROOT N1.5 on RoboCasa (100 demos).
What does the memory module memorize? We qualitatively analyze (a) how the proposed moment tokens encode information each timestep and (b) how the memory module processes past information. First, we confirm that the moment token attends more to task parts that change over timesteps and less to static parts. Indeed, as shown in Figure 4(a), higher attention values of the moment token concentrate on the gripper and objects that are associated with task success, while lower attention values are assigned to background regions. This is possibly due to the initialization via time-contrastive loss (see Section 3.1,, which encourages the tokens to extract distinguishable features over time. Next, we observe that the memory module selectively attends to past information depending the context within the episode. As shown in Figure 4(b), in the Cover-and-Stace task, at the moment when it is necessary to decide which cup to approach after the cube is covered with cup, the memory module assigns higher attention to the past timestep when the blue cube was previously visible. Additional qualitative results are provided in Section B.2.
Efficiency analysis. We analyze the efficiency of HAMLET by comparing with the multi-fram baseline which naïvely appends past observation frames, under varying history lengths. Specifically, we measure average latency (ms) and average peak GPU memory usage (MB) per environment timestep in the RoboCasa simulation. As shown in Table 4, the multi-frame baseline incurs substantial overhead in both metrics. For example, at a history length of 8, it requires roughly 2.4x greater latency and 7x higher memory than vanilla inference. In contrast, HAMLET shows only minimal overheads across history lengths, e.g., 1.02x at length 4 and 1.07x x at length 8. These results demonstrate the clear advantage of HAMLET over the multi-frame baseline in terms of efficiency.
8

---

Published as a conference paper at ICLR 2026
Table 4: Efficiency analysis. Average latency and peak memory usage measured on RoboCasa datasets. Both metrics are computed at each timestep within an episode and then averaged. For fair comparison, memory for original VLA parameters is excluded, except for the memory module in HAMLET. All measurements were on an NVIDIA A100 GPU. indicates lower values are better.
Method
History Length
Lateney (ms,
Peak memory (MB,
GROOT NL.5
80.5 (1.00x)
289 (1.00x)
+ Multi-frame
4
108.5 x1.35x)
1051 (3.64x)
HAMMLET (Ours)
4
82.4 (1.02x)
566 (1.96x)
+ Multi-frame
8
193.0 (2.40
2023 (7.00x)
HAMLET (Ours)
8
85.8 (1.07x)
578 (2.00x)
Table 5: Ablation study. Average success rate (%) on RoboCasa (100 demos) when selectively en- abling different components of HAMLET. Moment Concat. concatenates all moment tokens without a memory module, whereas the Transformer-based memory yields the best overall performance.
Moment Token
Memory Module
Token Length
Avg.
Method
Avg.
TCL
Avg.
1
64.3
No Memory
62.6
X
X
62.6
4
65.4
Moment Concat.
62.7
X
63.1
8
66.4
RNN
64.5
63.4
16
65.9
LSTM
65.0
X
64.8
32
62.7
GRU
64.3
65.4
64
62.5
Transformer
65.4
(a) Component analysis.
(b) Moment token length.
(c) Memory architecture.
Ablation study. We perform ablations on HAMLET to study the individual roles of its components, moment token length, and memory design. First, to assess the contribution of each component, we conduct ablation study by removing key components one at a time: (i) the use of moment tokens, (ii) the initialization of Cone-Contaattive Learning (TCL), and (iii) the presence of the memory module. In Table 5a, we confirm that removing the memory module leads to the largest performance drop, thereby confirming its critical role. We also observe that removing TCL-initialization consistently decreases performance. regardless of whether the memory module is used. Next, examine the effect of moment token length (default = 4). Table 5b shows the performance improves as the length increases up but then gradually decreases beyond this point. Lastly, to understand the effect of design choices for memory module, we compare several architectures: (i) Moment Concat, which naïvely concatenates past moment tokens without memory module, (ii) RNN (Graves, 2012), LSTM (Hochreiter & Schmidhuber, 1997), and GRU (Cho et al., 2014), which are recurrent variants that utilize a single accumulated hidden state, (iii) Transformer, which used in our experiments. From Table 5c, we find that the Transformer achieves the highest average success rate among the memory architectures. Interestingly, we observe that Moment Concat. yields almost no gains over the baseline, whereas other memory-based methods have shown performance gains.
Generalization capability. We investigate whether the memory module in our framework can transfer to unseen tasks, under the hy- pothesis that learns generalizable knowledge for identifying which information from past timesteps important, not limited to a specific dataset. To verify this, we first train the memory module and then freeze while evaluating on different datasets. Specifically, we first train the memory module on LIBERO and then transfer it directly to RoboCasa. As shown in Table 6. even in this setup, HAMLET achieves a success
Table 6: Generalization of memory module. The memory module is trained with the dataset left of the arrow. A LIBERO LIBERO-pretrained module provides gains for manipulation that the learned memory representations can gener- alize across embodiment datasets.
RoboCasa, identifying
Method
Avg.
GROOT NI.5
62.6
+ HAMLETT (LIBERO - RoboCasa)
64.5
+ HAMLET (RoboCasa + RoboCasa)
65.4
rate comparable the in-distribution setting, where both training and evaluation are performed on RoboCasa. This demonstrates the practical flexibility HAMLET to generalize across datasets, while eliminating the additional training cost for memory module adaptation.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this work, we address the limitation of existing Vision-Language-Aatte models (VLAs), which typically rely on the current observation while ignoring past context. We propose HAMLET, simple yet effective framework that enables pre-trained VLAs to leverage historical information without costly retraining from scratch. By introducing a lightweight memory module that integrates learnable moment tokens, HAMLET achieves significant improvements on long- long-horizon real-world tasks and standard simulation benchmarks, while maintaining computational efficiency. We hope our approach paves the way toward leveraging complex robotic manipulation tasks reliably and effectively in practice.
in off-the-shelf VLA policies to tackle more
REPRODUCIBILITY STATEMENT
We provide full hyperparameter and implementation details in Section and Section A.
ACKNOWLEDGMENTS
This work was partly supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RSS-019-II19000 Artifi- cial Intelligence Graduate School Program (KAIST); No. RS- 15-2202500253111 High-Performange Performance Research AI Computing Infrastructure Support the PFLOPS Scale; No. RS-2024-00509279, Global AI Frontier Lab). We are grateful to the RLWRLD Inc. for generously providing compute re- sources that supported significant portion of the experiments conducted in this work. We also thank Jimin Lee and Heeseung Kwon for providing helpful support on conducting real-world experiments.
REFERENCES
Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726 2024.
Johan Bjorck, Valts Blukis, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GrOOt n1.5: An improved open foundation model for generalist humanoid robots. https:/ hatp:://researdeee wnviiiiii...... com/
Labs/gesd gwwgggeeegrrrrrrooeeee
June 2025a. Accessed: 2025-09-09.
Johan Bjorek, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang. Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GrOOt nl: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.144 14734, 2025b.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. mo : A vision-language-actio flow model for general robot control. RSS, 2025.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PmLR, 2020.
Xi Chen, Josip Djolonga. Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Car- los Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, et al. Pali-x: On scaling up multilingual vision and language model. arXiv preprint arXiv:2305. 18565, 2023.
Cheng Chi, Zhenjia Xu. Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273668 2023.
Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decodee approaches. In Dekai Wu, Marine Carpuat, Xavier Carreras, and Eva Maria Vecchi (eds.), Proceedings of SSST-8, Eighth Workshop
10

---

Published as a conference paper at ICLR 2026
Syntax, Semanties and Structure in Statistical Translation, pp. 103-111, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/w1/W14-4012. URL https: //aclanthology.oo
Zichen Jeff Cui, Yibin Wang, Nur Muhammad Mahi Shafiullah, and Letrel Pinto. From play to policy: Conditional behavior generation from uncurated robot data. arXiv preprint arXiv:2210. 10047, 2022.
Pim De Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. Advances in neural information processing systems, 32, 2019.
Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378 2023.
Alex Graves. Supervised sequence labelling. In Supervised sequence labelling with recurvent neural networks, pp. 5-13. Springer, 2012.
Matthew J Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps. In AAAI fall symposia, volume 45, pp. 141, 2015.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9()): 1735-1780, 1997.
Huang Huang, Fangchen Liu, Letian Fu, Tingfan Wu, Mustafa Mukadam, Jitendra Malik, Ken Goldberg, and Pieter Abbeel. OTTER: A vision-language-acctoo model with text-aware visual feature extraction. In Forty-second International Conference on Machine Learning, 2025. URL https: wpw.revieweeeeeeeeeeeeeeeeeeewwe wtt.:viwwwwee.eeeeee...m
net/ Poorumiid-Une Ckm7R5k
Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-condittonee language models. In International Conference on Machine Learning, 2024.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. arXiv preprint arXiv: -XXv:1911.00172 2019.
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint ar arXiv:2403.1294. 2024.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source laaaonllanaag-aattooon model. In Conference on Robot Learning. 2024.
Seungjae Lee, Yibin Wang. Haritheja Etukuru, H. Jin Kim, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Behavior generation with latent actions. In International Conference on Machine Learning, 2024. URL https: /
net forum? id=hoVwecMoVS.
Barak Lenz, Opher Lieber, Alan Arazi, Amir Bergman, Avshalom Manevich, Barak Peleg, Ben Aviram, Chen Almagor, Clara Fridman, Dan Padnos, al. Jamba: Hybrid transformer-manee language models. In International Conference on Learning Representations, 2025.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented genera- tion for noolledge-nttensin nlp tasks. Advances in Neural Information Processing Systems, 33: 9459-9474, 2020.
Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: foundational vision- vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:241 I. 19650, 2024a.

---

Published as a conference paper at ICLR 2026
Xinghang Li, Peiyan Li, Minghuan Liu, Dong Wang, Jirong Liu, Bingyi Kang, Xiao Ma, Tao Kong, Hanbo Zhang, and Huaping Liu. Towards generalist robot policies: What matters in building siion-language-aee
models. arXiv preprint arXiv:2412.14058, 2024b.
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sich, Sean Kinmani, al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.00 0,944, 2024c.
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. NeurlPS, 014444766449999 2023.
Huaping Liu, Xinghang Li. Peiyan Li, Minghuan Liu, Dong Wang. Jirong Liu, Bingyi Kang, Xiao Ma, Tao Kong, and Hanbo Zhang. Towards generalist robot policies: What matters in building vision vision-language-action models. arXiv preprint 2rXiv:2412.14058, 2025.
Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and representation via value- value-implicit pre-training. arXiv preprint arXiv: 10.00030, 2023.
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601 2022.
NVIDIA Nemotron Nano. Efficient hybrid mamba-transformerr reasoning model. arXiv preprint arXiv:2508. 14444, 2025.
Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems, 2024.
Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x- x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration O. IEEE Interna- fional Conference on Robotics and Automation, pp. 6892-6903. IEEE, 2024.
Emilio Parisotto, Francis Song, Jack Rac, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayakumar, Max Jaderberg. Raphael Lopez Kaufman, Aidan Clark, Seb Noury, et al. Stabilizing transformers for reinforcement learning. In International Conference on Machine Learning, pp. 7487-7498.
PMLR, 2020.
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025.
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang. Jia Yuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual- langugg- action model. arXiv preprint arXiv:2501.15830 2025.
Seokin Seo, HyeongJoo Hwang, Hongseok Yang, and Kee-Eung Kim. Regularized behavior cloning for blocking the leakage of past action information. Advances in Neural Information Processing Systems, 36:21 (6:2128-21553 2023.
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time-contrastive networks: Self-supervised learning from video. In IEEE International Conference on Robotics and Automation, pp. 1 1134-111 1141. IEEE, 2018.
Nur Muhammad Shafiullah, Zichen Cui, Ariuntuya Arty Altanzaya, and Lerrel Pinto. Behavior transformers: Cloning & modes with one stone. In Advances in Neural Information Processing Systems, volume 35, pp. 22955-22968, 2022.
Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cogninee memory in lasion-language- action models for robotic manipulation. arXiv preprint arXiv:2508. /9236, 2025.
12

---

Published as a conference paper at ICLR 2026
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, al. End-to-end memory networks. Advances in Neural Information Processing Systems, 28, 2015.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. In Conference on Robot Learning, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprini arXiv:2307.09288. 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Homer Rich Walke, Kevin Black, Tony Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. Conference on Robot Learning, pp. 1723-1736 PMLR, 2023.
Chuan Wen, Jierui Lin, Trevor Darrell, Dinesh Jayaraman, and Yang Gao. Fighting copycat agents in behavioral cloning from observation histories. Advances in Neural Information Processing Systems, 33:2564-2575, 2020.
Siyu Xu, Yunke Wang, Chenghao Xia, Dihao Zhu, Tao Huang, and Chang Xu. Vla-cache: Towards efficient vision-language-action model via adaptive token caching in robotic manipulation. arXiv preprint arXivv2502.02115, 2025.
Yantai Yang, Yuhao Wang, Zichen Wen, Luo Zhongwei, Chang Zou, Zhipeng Zhang, Chuan Wen, and Linfeng Zhang. Efficientvla: Training-free acceleration and compression for vision-language- action models. arXiv preprint arXiv:2506. 10100, 2025.
Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine fine-graiedd bimanual manipulation with low-cost hardware. In Robotics: Science and Systems, 2023.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pp. 1165--183, PMLR, 2023.
13

---

Published as a conference paper at ICLR 2026
A EXPERIMENTAL DETAILS
A.1 MODEL DETAILS
In our experiments, we evaluate diffusion-based VLAs: GROOT NI (Bjorck et al., 2025b)', GROOT N1.5 (Bjorck et al., 2025a) (Black et al., 2025)', To-FAST (Pertsch et al., 2025)*, and Co- gACT (Li et al., 2024a)². All checkpoints are obtained from their official repositories.
A.2 DATASETS
We evaluate HAMLET across both real and simulation environments, as demonstrated Figure 5.
Real-World
Simulation
PnP Twice
Cover-and-Stack
Swap Cubes
RoboCasa
LIBERO
SimplerEnv
Figure 5: Evaluation environments. Left: Real-world evaluation comprises three tabletop tasks, which especially require awareness on historical context. Right: Simulation benchmarks include RoboCasa (Nasiriany et al., 2024) Kitchen, LIBERO (Liu et al., 2023), and SimplerEnv-Bridge (Li et al., 2024c), which consist of diverse indoor manipulation tasks.
Real-world environment. As shown in Figure 6, with a Robotiq 2F-85 gripper, following the DROID (Khazatsky al., 2024) setup. Two camera views provided: one mounted on the table and another the wrist. On this real-robot platform, we design three handcrafted tabletop tasks as illustrated in Figure 7: (i) aick--ad-Place Twice, where cube between two sides twice; (ii) Cover-and-Stack, where the robot
Franka Research 3 robot arm equipped
the robot with the nearest cup and then stacks another cup on top; and (iii) Swap Cubes, where the robot swaps the positions of two cubes using an auxiliary site. For each task, we collect 50 demonstrations for training, and report frame statistics in Table 7. Since trajectories are average about 268 frames long, we evaluate each trial with a maximum limit of 700 timesteps: if the task has been completed by then, the episode is counted as a failure.
cube
Zed Mini Wrist Stereo Camera
Adjustable Zed2 Stero Camera
Research 7DoF Robot Arm
Robotiq 2F-85 Gripper
Exterior
Wrist
(a) Front view
(b) Side view
(c) Camera views
Figure 6: Real-robot platform. We specify the robot specifications and multi-camera views.
14

---

Published as a conference paper at ICLR 2026
Pick-and-Puie -llce Twice
Partial
Full
Start!
Pick up cube
Move it opposite side
Release Pick up again
Place it back!
Cover-and -aack
Partial
Full
Start!
Pick up cup
Cover cube
Move to other cup Move opposite side
Stack it!
Swap Cubes
Partial
Full
Start!
Pick up blue cube
Stage it
Move green cube
Move blue cube!
Figure 7: Real-world tasks. We design three history-dependent real-world tasks. Each partial criteria
(Pick-and- Place Once, Cover Cube, Stage Cube) is described the corresponding frame.
Table Dataset statistics for real-world tasks. For each real-world dataset, we present the mean, maximum, and standard deviation of frame counts per episode. These statistics provide a basis for determining an appropriate maximum timestep limit during evaluation.
Task
Mean #frames
Max #frames
Std #frames
Pick-and-Place Twice
261
350
25.1
Cover-and-Stacck Swap Cubes
230
270
24.2
312
361
20.8
Simulation benchmarks. We evaluate HAMLET on three standard simulation benchmarks: (i) RoboCasa (Nasiriany et al., 2024), (ii) LIBERO (Liu et al., 2023) using GROOT N1.5, and (iii) SapplerEnn-Briidee (Walke et al., 2023) using CogACT. RoboCasa is a suite kitchen manipulation tasks, comprising 24 tasks across 120 scenes. Following Bjorek al. (2025a), split the dataset into training and evaluation sets, evaluating each task over 50 episodes. LIBERO is designed to benchmark multi-task and lifelong robot learning, and includes 40 tasks grouped into four suites (Spatial. Object, Goal, and Long). Both RoboCasa and LIBERO use a Franka robot, and we adopt their officially released code for training and evaluation. Lastly, for Coaplernn--rrdeee focused on real-to-sim simolamuttiion transfer, we use the Bridge V2 dataset (Walke et al., 2023) with Widow robot.
A.3 IMPLEMENTATION
Moment token and memory module. Across experiments, we set the sequence length of moment tokens to 4 and use a 2-layer Transformer as the memory module. For the Transformer implemen- tation, we adapt the LLaMA codebase from Hugging Face. To initialize the moment tokens via Time-Contrastive Learning (TCL), we construct positive pairs using combination of photometric distortions (brightness, contrast, and color jitter), Gaussian blur/noise, and small-patch occlusions applied stochastically to the same frame. Negative pairs are obtained by uniformly sampling frames
11am
15

---

Published as a conference paper at ICLR 2026
that are more than 16 timesteps apart within the same trajectory, corresponding to the action horizon. We use a contrastive temperature of 0.07 for all TCL-based pre-training.
Muame baseline. We follow the approach of multi frame policies (Team et al., 2024; Liu et al., 2025) in handling historical observations to construct a naive multi-frame baseline. For GROOT N1.5, concatenate past observation frames across the three original views, L.e,, left, right, and wrist. For CogACT, since the original VLM is designed to process only single view per timestep, we instead use its cognition token and concatenate it condition the action expert.
A.4 TRAINING DETAILS
Based on the official training recipes (Bjorck et al., 2025a; Li et al., 2024a), we fine-tune GROOT N1.5 and CogACT.
GROOT N1.5: Fine-tuned with the memory module for 60k steps using a batch size of 32 and learning rate of le-5, Initialization of moment tokens with TCL is performed for up to 30k steps with a batch size of 64 and the same learning rate. During fine-tuning, both the VLM and moment tokens are kept frozen.
CogACT: Fine-tuned with the memory module for 20k steps using a batch size of 32 and learning rate of 2e-5. Initialization of moment tokens with TCL is performed for up to 30k steps with batch size 64 and the same learning rate, but training is terminated at 10k steps once the loss converges. During fine-tuning, both the VLM and moment tokens remain trainable.
In both cases, we use default history length of 4. For all other baselines, including To. To -ASAT, and GROOT N1, we also follow their official training codes.
A.5 COMPUTATIONAL RESOURCES
GROOT N1.5: Experiments conducted on NVIDIA A100 80GB GPUs. Fine-tuning with the memory module 4 GPUs requires 16 hours, comparable the standard training time of ~~ 14 hours. Initialization of moment tokens with TCL adds 5 hours on 2 GPUs.
CogACT: Experiments is conducted on NVIDIA H200 141GB GPUs. Fine-tuning with the memory module on 4 GPUs take ~9 hours, compared to < hours for the standard setup. TCL-initialization requires ~9 hours on 2 H100 GPUs.
We provide summarization table detailing the modification costs (i.e., model parameters, inference time, and training time) in Table 8.
Table 8: Modification cost across different VLA types. We report three metrics: model parameters, inference time, and training time. inference was conducted on a single NVIDIA A100 GPU. Training was performed on 4 NVIDIA H200 GPUs, except for GROOT N1.5, whose VLM backhone was frozen during training and was therefore trained on NVIDIA A100 GPUs.
Method
Model parameters
Inference time (ms)
Training time (hours)
GROOT N1.5
2.72B
80.5 (1.00x)
14
+ HAMLET
2.86B
82.4 (1.02x)
16
CogACT
7.63B
229.6 (1.00x)
4
+ HAMLET
8.17B
234.0 (1.01x) x)
B MORE EXPERIMENTAL RESULTS
B.I QUANTITATIVE RESULTS
We provide additional quantitative results to better understand the main components of HAMLET and its generalizability across various scenarios. We also present the complete RoboCasa metrics in Table 9, including the per-task success rates summarized in Table 2 for HAMLET and GROOT N1.5.
16

---

Published as a conference paper at ICLR 2026
Table 9: Full results RoboCasa Kitchen with different dataset size.
GROUT NI.5
GROOT N1.5 + HAMLET
Task (PnP= Pick-and-Placee
30 demos
100 demos
300 demos
30 demos
100 demos
300 demos
Close Double Door
80.0
90.0
82.0
82.0
Close Drawer
94.0
97.0
96.0
100.0
96.0
100.0
Close Single Door
94.0
98.0
94.0
96.0
Coffee Press Button
88.0
83.0
76.0
84.0
90.0
Coffee Serve Mug
62.0
63.0
$8.0
70.0
68.0
76.0
Coffee Setup Mug
24.0
29.0
28.0
30.0
28.0
26.0
Open Double Door
70.0
76.0
84.0
68.0
86.0
96.0
Open Drawer
40.0
71.0
68.0
74.0
68.0
Open Single Door
54.0
69.0
78.0
58.0
82.0
84.0
PnP from Cab Counter
22.0
$1.0
50.0
26.0
46.0
54.0
PnP Counter Cab
36.0
49.0
50.0
48.0
56.0
PnP from Counter Microwave
26.0
20.0
26.0
20.0
30.0
16.0
PnP from Counter Sink
38.0
60.0
52.0
50.0
56.0
42.0
PnP from Counter Stove
44.0
$2.0
68.0
24.0
54.0
64.0
PnP from Microwave Counter
22.0
40.0
44,0
24.0
24.0
24.0
PnP from Sink Counter
60.0
60.0
74.0
54.0
56.0
$6.0
PnP from Stove Counter
38,0
70.0
66.0
56.0
74.0
76.0
Turn Off Microwave
62.0
96.0
96.0
88.0
98.0
100.0
Turn Off Sink Faucet
68.0
84.0
84.0
76.0
88.0
90.0
Turn Off Stove
12.0
18.0
30,0
12.0
20.0
28.0
Turn On Microwave
42,0
44.0
48,0
50.0
60.0
72.0
Turn On Sink Faucet
46.0
83.0
70.0
66.0
76.0
86.0
Turn On Stove
30,0
$7.0
42.0
26.0
68.0
40.0
Turn Sink Spout
34.0
56.0
62.0
48.0
78.0
72.0
Pick-and-Plin
50.3
53.8
48.5
48.5
Open-or-Close
81.2
85.7
71.3
85.7
87.7
Others
61.3
59.4
66.8
68.0
Average
47.8
62.6
64.1
52.5
65.4
66.4
Table 10: Positive and negative pair construction for different TCL initialization methods. We report the positive and negative pair constructions used by SimCLR, single-view TCN, multi-view TCN, and our TCL initialization.
Method
Positive pair
Negative pair
SimCLR
Same view with classical image augmentations
In-batch samples
Single-view TCN
Same view a nearby timestep
Same view at far timestep Same view at far timestep Same view at far timestep
Multi-view TCN
Different view at the same timestep
TCL
Same view with classical image augmentations
Time-Conte Cone-Contrasttve Learning (TCL), To understand the impact of individual parameters in TCL., conduct an ablation study on different ways of constructing positive and negative pairs: (i) random initialization, (ii) SimCLR (Chen al., 2020), (iii) single-view TCN (Sermanet al., 2018), (iv) multi-view TCN (Sermanet et al., 2018), and (v) our TCL (default). Each method constructs positive and negative pairs from observation images, as summarized Table 10. Table 11 shows that our TCL initialization achieves the highest average success rate among all methods, highlighting its effectiveness as a core component.
We further provide more extensive analysis examining whether TCL. influences final performance across datasets. As shown in Table 12, TCL. consistently yields improvements across datasets and even provides substantial gains in low-data scenarios (i.e., up to +2.7% in RoboCasa 30 demos). These results highlight that TCL initialization plays crucial role enabling the moment tokens to effectively extract compact state features, especially when combined with other components such as the memory module.
More longer horizon scenarios. To validate the effectiveness of HAMLET in more complex scenarios, we further evaluate it on a longer-horizon task, Pick-and Place Three Times, which requires the model to return the ready position after performing three consecutive pick-and-place operations.
17

---

Published as a conference paper at ICLR 2026
Table 11: Analysis on different TCL initialization methods. We compare different TCL initial- ization by constructing positive and negative pairs. Our TCL method achieves the highest average success rate among all methods, demonstrating its effectiveness as core component.
RoboCasa Kitchen
LIBERO
Method
100 demo
Spatial
Object
Goal
Long
Avg.
Random initialization
64.8
99.4
99.8
98.0
87.8
96.2
SimCLR
63.8
99.4
98.6
97.0
87.2
95.5
Single-view TCN
64.8
98.6
99.4
98.2
91.6
96.9
Multi-view TCN
64.9
99.0
99.0
90.2
96.5
TCL (Ours)
65.4
99.0
100.0
99.2
92.2
97.7
Table 12: Effecc of TCL. We analyze the effect of TCL on RoboCasa Kitchen and LIBERO, For RoboCasa Kitchen, we report the average success rate (%) across 24 tasks with models trained using 30, 100, or 300 demonstrations per task. For LIBERO, each metric is the average success rate (%) across 10 tasks per suite, with training performed jointly on all suites. Bold indicates the best.
RoboCasa Kitchen
LIBERO
Moment Token
Memory Module
TCL
30
100
300
Spatial
Object
Goal
Long
Avg.
x
x
47.8
62.6
64.1
98.1
99.4
97.2
87.8
95.6
X
/
49.8
64.8
65.1
99.4
99.8
98.0
87.8
96.2
/
/
52.5
65.4
66.4
99.0
100.0
99.2
92.2
97.7
In particular, the policy needs to track how many successful pick-and-place operations have already been completed, requiring it memorize retains only 4 steps by default. As shown in Table 13, HAMLET achieves a significant improvement over the baseline on this long-horizon task, despite the required temporal dependency exceeding the original window size. This is possibly due to the causal nature of Transformers ( (Vaswani et al., 2017): the key-value pair of each token participates influencing their hidden states and the corresponding key-value entries. Consequently, even after past V-ecchhe entry is dropped, its information has already been integrated into the K V-cache of subsequent steps.
than 14 inference steps, whereas method explicitly
the self-attention computation for all later tokens,
Results on autoregressive VLAs. We further validate the generalizability of HAMLET on au- toregressive VLAs. Specifically, we apply it to OpenVLA (Kim et al., 2024) and -F-AA,, two representative autoregressive VLAs built on action discretization. We evaluate Open VLA on the LIBERO benchmark, while To FAST is evaluated on the RoboCasa Kitchen benchmark (100 de- mos). As shown Table 14, HAMLET consistently improves success rates over both baselines: for Open VLA, achieving 9.0% gain on LIBERO-Long, and for no -AST, achieving 5.8% gain on Pick-and-Place (PnP) tasks RoboCasa. These results highlight that HAMLET effectively transforms pretrained VLA into history-aware policies without relying any specific architecture. For implementation. we fine-tune both models using batch size of 32 for Open VLA and 64 for -o-FAST following their official repositories" for other details.
B.2 QUALITATIVE RESULTS
We provide additional qualitative results to better understand the main components of HAMLET.
Time-Contrastive Learning (TCL). To analyze the effect of Time-Contrastive Learning (TCL) on the moment token, we visualize the self-attention maps on RoboCasa, comparing randomly initialized tokens with TCL-initialized ones. As shown in Figure randomly initialized moment tokens attend to sparsely distributed regions, including background areas that remain static across timesteps. In contrast, TCL-initialized moment tokens focus on more task-relevant regions related to successful
Phttps: httpg::////ugingaaccee 'https:// thub. com/ comm/hhysccal-a Intel opencc//penppi
a/ppp//lallaa//////am//amm a-
18

---

Published as a conference paper at ICLR 2026
Table 13: More longer horizon scenarios. We validate HAMLET the longer-horizon task Pick-and-Place Three Times, which requires tracking three consecutive pick-and-place cycles. We report the success rate (%, over 24 trials per task) and the average number of executed pick-and-place operations. Bold indicates the best.
Method
Success rate
Executed PnPs
GROOT N1.5
8.3
1.042
+ HAMLET
37.5
1.958
Table 14: Results autoregressive VLAs. We evaluate HAMLET two representative autore- gressive models: (a) OpenVLA on the LIBERO benchmark, and (b) To-FAST on the RoboCasa Kitchen benchmark (100 demos). For LIBERO, tasks per suite, with training performed separately on each suite. For RoboCasa Kitchen, we report the average success rates (%) over 24 tasks, which are grouped into three categories: Pick-and-Plas (PnP), Open-or- -Cro-e (OoC), and Others. Bold indicates the best.
report the average success rate (%) across 10
(a) VLA
VLA (Kim al., 2024)
(b) T0-FAST (Pertsch al., 2025)
LIBERO
RoboCasa Kitchen
Method
Spatial
Object
Goal
Long
Method
PnP
OoC
Others
Open VLA
84.8
62.6
71.6
48.6
To-FAST
17.0
60.7
46.6
HAMLET
85.6
70.8
78.2
57.6
HAMLET
22.8
63.7
52.0
execution. For example, observe that the object to be grasped by the gripper tends to receive higher attention values than other regions.
Memory module. As explored in Figure 4, we further provide examples of how our memory module attends to historical context. Specifically, we visualize the attention maps of the memory module during rollouts in the Swap Cubes task, where the robot swaps the blue cube with the green cube. In the left side of Figure 9, we confirm that the memory network attends to the key past timestep, which is crucial for determining the next action: for example, when deciding which cube pick next, the memory module highly attends to the past timestep when the blue cube was previously placed down. Interestingly, the attention across past timesteps remains low when historical information is less critical, e.g., during the initial move of the blue cube, as shown on the right side of Figure 9.
Further example rollouts of real-world tasks. We further illustrate rollouts by HAMLET and GROOT N1.5 for each real-world task in Figure 10, 11, 12. It can be clearly observed that GROOT N1.5 without history- history-awareness struggles with multi-step dependencies and often fails to recover from occlusions ambiguous intermediate states, while HAMLET leverages its memory to better recognize the current state and complete tasks. In Figure 13, we further provide typical failure cases of naïve multi-frame baseline on GROOT N1.5, which often proceeds to the next actions while failing to recover from its failure state. This indicates that simply appending molti-frame inputs to VLAs might degrade their generalizability.
C DISCUSSION
Limitations. While HAMLET maintains the efficiency of single-frame VLAs through its efficiency- oriented design, it still incurs additional training cost due to initialization with time-contrastive learning. Furthermore, although we demonstrate its applicability to recent diffusion-basee VI.As, it may not directly extend to auto-regressive VLAAs (Pertsch al., 2025; Kim et al., 2024).
Future directions. One promising direction is to scale up HAMLET using larger-scale robotic manipulation datasets. As discussed Table 6, each components of HAMLET, such as the memory module, can transfer effectively to unseen datasets. This suggests that initializing these components on large-scale datasets and then rapidly adapting them further enhance the practicality of our framework. In addition, while adopted relatively shallow Transformer for memory module to ensure scalability, a more exhaustive search over architectural parameters or the use recent hybrid models (Lenz et al., 2025; Nano, 2025) is straightforward approach for obtaining additional gains.
19

---

Published as a conference paper at ICLR 2026
Inpu
w/o TCL
w TCL
Figure 8: Qualitative results of the moment tokens. We analyze the self-attention of moment tokens on RoboCasa imput frames, comparing random initialization (w/o TCL) with TCL-initialization (w/ TCL). After TCL-initialization, the moment tokens tend to concentrate task-relevant regions, such as the object to be grasped.
Blue cube placed down!
Blue cube picked up!
Recall
Wrist Exterior
Exterior
Wrist
Wait, which cube to grasp next?
Next action: Place down
Figure 9: Qualitative results the memory module. We analyze the self- of the memory module on the Swap Cubes task, where the robot swaps the blue cube with the green cube. We report two distinctive cases: (left) one where recalling specific past timestep is necessary for the next action-such as when the blue cube has already been placed down before moving the green oube-and (right) another where recalling is less critical-such as during the initial move of the blue cube. We observe that our memory module clearly attends to the relevant past timestep when required.
D THE USE OF LARGE LANGUAGE MODELS
We acknowledge that large language models (LLMMs) were used the preparation of this manuscript to assist with writing quality. We mainly utilize them to find grammatical errors, suggest alternative vocabulary. All ideas, analyses, and conclusions presented in this paper are solely those of the authors.
20

---

Published as a conference paper at ICLR 2026
Pick-and-Place Twice (left)
Stuck, confused where to go...
GROOT N1.5
Properly released!
HAMLEY ET
Pick-and-Place Twice (right)
Should release, but carries away...
GROOT N1.5
Properly released!
HAMLET
Figure 10: Example rollouts of Pick-and-Plas Twice task.
Cover-and -tank (left)
Grasps the same cup again...
GROOT N1.5
Properly reaches the opposite!
HAMLE ET
Cover-and-Stack (right)
Approaches the same cup again...
GROOT N1.5
Properly reaches the opposite!
HAMLET
Figure 11: Example rollouts of Cover-and-Stack task.
21

---

Published as a conference paper at ICLR 2026
Swap Cubes (blue on the left)
Approaches wrong cube...
GROOT N1.5
Placed the correct cube!
HAMLET
Swap Cubes (blue on the right)
Should release. but takes it back...
GROOT N1.5
Properly released!
HAMLET
Figure 12: Example rollouts of Swap Cubes task.
Pick-and-Plast Twice
H Failed to grasp!
But simply proceeds without recovery...
GROOT N1.5 Multi-frame
Cover-and ander-and-Stck
Failed grasp!
Swap Cubes
t Failed grasp!
Figure 13: Example rollouts by naive multi-frame baseline. We observe that the nave multi-frame baseline often proceeds without recovery, merely copying action trajectories observed during training. This is possibly due to its poor generalizability, using only consecutive frames throughout training.
22


Reviews:
[{"review_id": "DxOFlw3unq", "reviewer": "Reviewer_yS5A", "summary": "This paper proposes HAMLET, a plug-and-play framework to add history-awareness to pre-trained VLAs without costly retraining. It introduces \"moment tokens\" to compress each timestep and a lightweight memory module to aggregate them. The tokens are notably initialized with Time-Contrastive Learning (TCL) to focus on dynamic cues. The method proves highly efficient and effective on medium-horizon tasks, outperforming naive multi-frame baselines.", "questions": "See above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The framework is a highly efficient, plug-and-play module that demonstrates a significant advantage in computational cost (latency and memory) over naive multi-frame approaches.\n\n- The method is validated extensively across multiple VLA backbones (GROOT N1.5, CogACT) and benchmarks (real-world, RoboCasa, LIBERO), proving its generalizability and effectiveness.", "weaknesses": "- The memory mechanism is a simple rolling-window Transformer, which is not scalable for truly long-horizon tasks as critical information will eventually be dropped from the fixed-size history, which does not consider the history before the rolling-window.\n- The core methodological contribution of Time-Contrastive Learning (TCL) for token initialization provides only a marginal performance gain (0.6% in Table 5a) over a random initialization, questioning its overall necessity.\n- The comparison against the 'Multi-frame' baseline is a weak strawman, as this naive frame-stacking approach is known to fail and interfere with current-state grounding; the paper lacks benchmarks against more sophisticated history integration techniques.\n- The \"plug-and-play\" claim is misleading, as the framework requires a dedicated finetuning stage and a separate token initialization phase, which contradicts the common understanding of a plug-and-play module as a zero-shot, training-free component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jn48gvMzZK", "reviewer": "Reviewer_oHKU", "summary": "This article addresses the limitation of existing Vision-Language-Action models (VLAs) that rely solely on current observations while ignoring historical context. It proposes the HAMLET framework, which transforms VLAs into history-aware policies by compressing perceptual information at each timestep using **moment tokens** initialized via time-contrastive learning (TCL), and integrating these historical moment tokens to generate memory features through a lightweight Transformer memory module. The article conducts experiments in both real-world (three tabletop tasks) and simulation benchmarks (RoboCasa Kitchen, LIBERO, SimplerEnv-Bridge). Results show that HAMLET, built on GR00T N1.5, achieves an average success rate of 76.4% on history-dependent real-world tasks (surpassing the baseline by 47.2%), improves performance from 64.1% to 66.4% on RoboCasa Kitchen (100-demo setup), and from 95.6% to 97.7% on LIBERO. Additionally, it incurs only minimal computational overhead (1.02× latency and 1.96× peak memory usage at a history length of 4) and can be adapted to different VLA backbones such as GR00T N1.5 and CogACT, verifying the framework’s effectiveness and generalizability.", "questions": "see weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The article accurately identifies the core contradiction in robotic manipulation tasks—VLAs’ \"single-frame dependency\" versus the \"history-dependent nature of tasks\"—and addresses the pain point of high computational overhead in traditional multi-frame baselines. It directly responds to the demand for efficient history-aware policies in the field of robot learning, demonstrating clear research value.\n\n2. The combined design of moment tokens and a lightweight memory module is ingenious. Moment tokens reduce redundant information storage through compression, while the Transformer-based memory module enables selective integration of historical information. Notably, the framework does not require large-scale retraining of pre-trained VLAs, meeting the practical \"plug-and-play\" requirement. The technical route is clear and innovative.\n\n3. Experiments cover both real-world and simulation environments, including comparisons across multiple tasks, baselines, and VLA backbones. Further analyses—such as efficiency evaluations, ablation studies (on component contributions, token length, and memory architecture), and cross-dataset transfer experiments—validate the framework’s performance, efficiency, and generalizability. The detailed data and logical reasoning enhance the credibility of the conclusions1", "weaknesses": "1. Currently, HAMLET uses moment tokens (default length of 4) and a lightweight 2-layer Transformer memory module, which perform well in the article’s designed tabletop tasks and standard kitchen manipulation tasks. However, validation is lacking in more complex scenarios, such as multi-object interactions, dynamically disturbed environments, or long-horizon tasks (with a history length exceeding 8). The mentioned issue of \"the lightweight historical feature compression network\" is valid: while the lightweight design ensures efficiency, it may insufficiently capture historical information due to limited feature expression capabilities in complex scenarios. Additional experiments are needed to demonstrate the framework’s robustness in such contexts.\n\n2. The article only verifies HAMLET’s performance on diffusion-based VLAs (GR00T N1.5, CogACT) and does not extend it to autoregressive VLAs (e.g., models proposed by Pertsch et al., 2025; Kim et al., 2024). It also fails to analyze the modification costs and performance differences when adapting the framework to different VLA types, limiting the breadth of arguments for the framework’s applicable scope.\n\n3. the article mentions that TCL enhances the temporal discriminability of moment tokens but does not elaborate on how specific parameters of TCL—such as the intensity of augmentation methods and negative sample selection strategies—influence token quality. It also lacks visualizations or quantitative analyses comparing the feature expression capabilities of moment tokens under different initialization methods (e.g., random initialization, other self-supervised approaches), leading to insufficient justification for the design of this core component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pVCiQxbM1G", "reviewer": "Reviewer_JGoo", "summary": "This paper introduces HAMLET, primarily addressing the memory challenges inherent in Vision-Language-Action (VLA) models. The authors propose the use of Memory Tokens combined with a time-contrastive learning objective applied to the output features. This approach aims to obtain a compressed representation of the VLA's current internal state. This compact feature can then be provided to an action expert, supplying essential historical information. The method is notably lightweight and efficient, achieving a significant reduction in memory overhead.", "questions": "The authors successfully demonstrate and test many real-world tasks that are heavily dependent on memory, where the performance improvement brought by HAMLET is expected. However, the method also yields performance gains on benchmarks like LIBERO and SIMPLER. Since these tasks are logically completable without reliance on historical information, why does HAMLET still provide a performance boost? Could the authors offer an analysis or hypothesis for this unexpected improvement on tasks that are ostensibly memory-agnostic?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper is easy to follow, and the figures are clear and highly informative.\n\n- The motivation behind this work is compelling. Most current VLA models neglect the issue of memory, and their tested benchmarks are often simple, placing minimal demands on historical information. However, as the authors correctly illustrate, real-world tasks requiring memory are abundant. Addressing this class of problems is crucial for VLA deployment.\n\n- The proposed method is simple yet effective, incurring a low training burden. Crucially, it can be easily applied to many existing pre-trained VLA models to significantly enhance their memory capabilities.", "weaknesses": "This paper is comprehensive and well-executed; I found no discernible weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AubngKLh2H", "reviewer": "Reviewer_rLRn", "summary": "The paper observes that multi-step robotic manipulation tasks are often history-dependent, whereas most existing Vision-Language-Action (VLA) models rely only on the current observation. To address this, the authors propose HAMLET, a framework that transforms pretrained VLAs into history-aware policies by introducing moment tokens, compact temporal representations learned through time-contrastive learning, and a lightweight memory module that aggregates past tokens to enhance action prediction.", "questions": "1. Using past information is a very common strategy, as many previous imitation learning methods have incorporated historical observations [1,2]. Therefore, this aspect of the work feels somewhat less novel. However, the impressive results largely compensate for this. I am genuinely curious—why does adding the query token and aggregating past information lead to such a significant improvement? It would be great if the authors could provide more analysis on this.\n\n2. As discussed in [1] and several other works, incorporating past information can lead to the copycat problem. Did you encounter this issue in your experiments?\n\n3. In the simulation experiments, the multi-frame setting performs worse than the baseline. Why is that the case? Intuitively, having access to more information should improve performance, shouldn’t it?\n\n[1] Fighting Copycat Agents in Behavioral Cloning from Multiple Observations. NeurIPS 2023.\n\n[2] LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. NeurIPS 2023.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The experimental results are very impressive.\n\n2. This is a simple yet highly effective approach.\n\n3. The writing is clear and well-structured.", "weaknesses": "See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)