VITA: Zero-Shot Value Functions via Test-Time Adaptation of Vision–Language Models
Authors: Christos Ziakas
Alessandra Russo
Venue: ICLR 2026 Poster (2026)
Keywords: Meta-Learning
Test-Time Adaptation
Value Function Estimation
Vision-Language Models
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: VL-TTA enables generalizable zero-shot value function estimation through test-time adaptation of contrastive VLMs

Abstract: Vision–Language Models (VLMs) show promise as zero-shot goal-conditioned value functions, but their frozen pre-trained representations limit generalization and temporal reasoning. We introduce VITA, a zero-shot value function learning method that enhances both capabilities via test-time adaptation. At inference, a lightweight adaptation module is updated via a gradient step on a meta-learned self-supervised loss, such that each test-time update improves value estimation. By updating sequentially over a trajectory, VITA encodes history into its parameters, addressing the temporal reasoning limitations. To mitigate shortcut learning, we propose a dissimilarity-based sampling strategy that selects semantically diverse segments of the trajectory during training. In real-world robotic manipulation tasks, VITA generalizes from a single training environment to diverse out-of-distribution tasks, environments, and embodiments, outperforming the state-of-the-art zero-shot method using autoregressive VLMs. Furthermore, we demonstrate that VITA’s zero-shot value estimates can be utilized for reward shaping in offline reinforcement learning, resulting in multi-task policies on the Meta-World benchmark that exceed the performance of those trained with the simulation’s fuzzy-logic dense rewards. Project website: https://chziakas.github.io/vita/.

Published as a conference paper at ICLR 2026
VITA: ZERO-SHOT VALUE FUNCTIONS VIA TEST-TIME ADAPTATION OF VISION-LANGUAGE MODELS
Christos Ziakas" Department of Computing Imperial College London
Alessandra Russo Department of Computing Imperial College London
ABSTRACT
Vision-Language Models (VLMs) show promise as zero-shot goal-conditioned value functions, but their frozen pre-trained representations limit generalization and temporal reasoning. We introduce VITA, zero-shot value function learning method that enhances both capabilities via test-time adaptation. At inference, a lightweight adaptation module is updated via gradient step a meta-learned self-supervised loss, such that each test-time update improves value estimation. By updating sequentially over trajectory, VITA encodes history into its parameters, addressing the temporal reasoning limitations. To mitigate shortcut learning, we propose dissimilarity- based sampling strategy that selects semantically diverse segments of the trajectory during training. In real-world robotic manipulation tasks, VITA generalizes from single training environment diverse out-of-distribution tasks, environments, and embodiments, outperforming the state-of-the-art zero-shot method using autoregressive VLMs. Furthermore, we demonstrate that VITA's 's zero-shot value estimates can be utilized for reward shaping in offline reinforcement learning, resulting in multi-task policies on the Meta- World benchmark that exceed the performance of those trained with the simulation's 's fuzzy-logic dense rewards. Project website: https:: //tp://cchiiaee htccchiiaat iakas. Backkas.gtthuke github. io/vita/
INTRODUCTION
Vision Vision-Language Models (VLMs) have demonstrated strong generalization across diverse tasks and domains by learning from large-scale, unstructured web data without human supervision (Radford
et al. 2021 Alayrac al. 2022). In contrast, despite significant advances in learning generalist policies for robotics (Brohan et al. 22023) Kim et al. 2024) and 3D virtual environments (Reed et al.
2022 SIMA Team et al.) 2024), state-of-the-art methods have yet achieve comparable success due
to their reliance on expert demonstrations (Ho & Ermon 2016). World models (Ha & Schmidhuber
2018 Hafner et al., 2021 Matsuo et al., 2022) have emerged as promising solution, enabling
agents to learn in simulated environments, with applications spanning robotics ( (Yang et al. 2023
Agarwal et al. 2025). autonomous driving (Hu et al. 2024), and video games (Bruce et al., 2024).
Despite the potential of world models to generate realistic visual trajectories in diverse, open-ended
environments (Hughes et al. 2024, Silver Sutton 2025). a critical challenge remains: how can
agents effectively learn from videos at scale without relying human supervision?
line of research explores policy learning directly from expert visual trajectories by inferring actions from visual transitions (Edwards et al. 2019: Baker et al.) 2022 Schmidt & Jiang 2024 Zhang jtt al. 2025a). These latent actions can be mapped to executable controls in deployment, although this remains a challenging task in practice. In parallel, other work focuses on learning a universal goal-conditioned value function, using it as a zero-shot reward shaping and supervision signal for reinforcement learning (RI.) and imitation learning, respectively (Chen et al.) 2021 Ma et al., (022). In this framework, goal-conditioned value estimation can be formulated as: predicting how far an agent has progressed toward completing a task, based on visual observations and natural language
task description (Lee et al. 2021a Ma et al. 2023b 2024 Dashora et al.) 2025).
Prior work has employed pre-trained contrastive VLMs (Radford et al.) (021) for zero-shot reward shaping, leveraging the similarity between task descriptions and visual observations in a shared
"Coreesonning author: c. zi akas24@ imperiaa
ac.uk

---

Published as a conference paper at ICLR 2026
multimodal representation space to ground progress in semantic context (Ma et al. 2023b Baumli et al. 2023; Rocamonde et al. 2024). However, these methods fail to capture the temporal context needed to disambiguate visually similar states that occur at different stages of task (e.g., folding vs. unfolding a shirt). In contrast, autoregressive VL.Ms (Alayrac et al. 2022) incorporate temporal context by conditioning on the entire visual trajectory within the prompt: however, they inherit bias toward monotonically increasing predictions from the chronologically ordered datasets used in pre-training (Ma al. 2024). Both VLM architectures rely on pre-trained representations (Ma) et al., 2023b 2024] (Bamli et al. 2023; Rocamonde et al. 2024, Du et al. 2023) for zero-shot prediction, which limits both their generalization and their temporal reasoning (Päträucean al., 2023). To enable generalizable value function learning, recent work has explored large-scale pre- training (Alakuijala et al. 2024; Ma et al. 2023b), domain-specifie fine-tuning (Zhang et al. 2025b).
and demo-based reward specification (Sontakke et al. 2023).
In this work, we introduce VITA, a zero-shot value function learning method that enhances both generalization and temporal reasoning through test-time adaptation, outperforming the state-of-the-art zero-shot approach (Ma et al. 2024) on real-world robotic manipulation tasks. Unlike prior methods that rely large-scale tre-training expert demonstrations for generalization, VITA adapts online to both the semantic and temporal context of trajectories. At inference, a lightweight adaptation module is updated in negligible time with gradient step on a meta-learned self-supervised loss (Sun) eet al. 2024), such that each test-time update improves value function estimation (Finn et al. (017))
By updating sequentially over a test trajectory, the value function estimator encodes trajectory history into its parameters (Sun et al. 2024). thereby addressing the temporal reasoning limitations of prior work. To mitigate shortcut learning (Geirhos et al.) 2020), we introduce a dissimilarity-bate sampling strategy that encourages reliance on semantic cues, supported by empirical evidence. Our evaluation demonstrates the effectiveness of our method across core capabilities of value function estimation: generalization under distribution shifts, differentiation between expert and non-expert trajectories, and reward shaping for offfine RL (Levine et al. 2020).
We summarize our key contributions as follows:
We propose VITA, a test-time adaptation method that enhances the generalization and temporal reasoning of contrastive VLMs for zero-shot value function estimation, without requiring task-specific demonstrations or large-scale pre-training.
VITA generalizes from single training environment to diverse -fistiiiiiittnoonnn tasks, environments, and embodiments in robotic manipulation, outperforming the state-of-the-a zero-shot method (Ma al. 2024).
VITA's zero-shot value estimates for reward shaping yield offline RL policies the Meta- World benchmark for multi-task learning (MT10) that surpass those trained with the simulation's 's fuzzy-logic dense rewards (Yu et al. 2020).
2 PRELIMINARIES
2.1 GOAL-CONDITIONED VALUE FUNCTIONS
In video trajectories. task progress estimation can be viewed as goal-conditioned value function estima- tion, where the reward reflects task completion. Therefore, we formulate learning goal-conditiones value function as predicting the degree of task completion from languaee representations. Formally, the value function is defined as: V O x G + [0, 1] which maps an observation E O and a goal specification g E 9 to a scalar value indicating the predicted progress toward goal com- pletion. We set V(o,;g) g) 0 to correspond to the start of the task and V(o.g) = to indicate completion. Task progress is commonly aligned with temporal position in expert demonstrations, based on the assumption that such trajectories exhibit monotonically increasing progress toward goal completion (Lee et al. 2021a; Ma et al.) 2024 Dashora et al.] (2025). Given an expert tra- jectory (01, (01...., Or) Tg, temporal progress is defined using normalized timestep indices:
V (o,: g) = where T is the trajectory length. Therefore, temporal progress provides supervision for learning a goal-conditioned value function V.
2

---

Published as a conference paper at ICLR 2026
Vl pred
pellie
LangunLanguage Encoder
Adaptation Module
Value Function
l pred
Head
0.-1 Volaet 0c-1)
(a) Training: Learning yoal-conditioned value function estimator via meta- meta-learning
Foldd clush
Visia--I anguage Eucader
Adaptation Module
Value Funcrion
01
50%
100%
0, 0
Voluer Ze; O
(b) Inference: Zero-shot value estimate for OOD test trajectory via test-time adaptation
Figure 1:
Overview of VITA. VITA learns goal-conditioned value function via meta-leate weta-learning and achieves zero-sin shot generalization out-of-distribution trajectories via test-time adaptation.
2.2 TEST-TIME TRAINING FOR SEQUENCE MODELING
Test-time training (TTT) (Sun al. 2020 test-time adaptation method that treats inference as self-supervised learning task (Goodfellow et al. 2016), updating model parameters each test instance without access to labels. While originally proposed for static image classification (Sun)
et al. 22200), recent work extends TTT to sequence modeling (Sun et al. 2024 Wang al. 2023). where a parametric adaptation module falups is updated at each timestep using gradient step on self-supervised loss. In contrast to standard sequence architectures that encode temporal history in activation states, such as in hidden states (Hochreiter & Schmidhuber 1997| Cho et al., 2014) or in cached key-value pairs (Vaswani al. (017). TTT encodes history into the parameters of addgpt via sequential updates in inference. The parameters of Satup thereby serve as an implicit memory, encoding temporal history while preserving temporal order. [Sun al. (2024) further propose to meta-learn the self-supervised task, following the gradient- -basee meta-learning paradigm (Finn et al. 2017). In this setting (Sun et al. 2024). the self-supervised loss parameterized by learnable linear projections and trained to minimize downstream prediction loss after test-time update, rather than directly minimizing In this work, we apply TTT to coal-conditioned value function learning, where temporal context is captured through sequential updates, and the self-supervised task is meta-learned to improve value estimation rather than being predefined a priori.
VITA: ZERO-SHOT VALUE FUNCTIONS VIA TEST- TIME ADAPTATION
3.1 MODEL ARCHITECTURE
Our goal-conditioned value function estimator comprises three modules: (1) multimodal encoder that extracts joint visual-language representations from visual trajectories and their task descriptions; (2) adaptation module updated at test-time using self sule-supe loss that is meta-leamed to improve value estimation; and (3) regression head that predicts value estimates.
3.1.1 MULTIMODAL INPUT REPRESENTATION
We use a frozen contrastive vision- language encoder, CLIP (Radford al. 2021 p, extract repre- sentations from visual observations and goal descriptions. Given visual trajectory (01..... or) and a language task description g. we concatenate their representations at each timestep to form sequence of joint multimodal representations (21, =2, ... 2T), where 2; (o: &, (g) e g and de and dy denote the visual and language encoders, respectively. CLIP is pre-trained with contrastive objective to align paired image-text inputs in shared representation space (Radford
(et al.) 2021). As a result, representations of visual observations that are semantically closer to goal completion tend be closer to the representations of the goal description.

---

Published as a conference paper at ICLR 2026
Zt
P
f adapt
(:; 1--1)
Pp
Lolif
[ adapee (Pdape(Px2ze (Px 0(-1) Przzll 11
"Fold the cloth from left right
Φ g
0, 01-1 17 VVolsott Zei ①c-1)
PQ Q
f Fddapt
(.;0t)
h
Value Fanction Estimation: 50%
Test-time Adaptation. In inference, at each timestep f, an adaptation module fadape is updated via a gradient step on meta-learned self-supervised loss leetr encoding temporal history.
Figure 2:
3.1.2 TEST-TIME ADAPTATION
To adapt multimodal representations both semantic and temporal context, employ an adaptation module fadape following the test-time training paradigm described in 2.2 At inference, the parameters of fadope are updated online via gradient step on a meta-learned sell-supervised loss &eet- This loss is formulated as reconstruction objective using learnable linear projections Pk and Py, which are meta-learned during training to improve value estimation after test-time adaptation. In particular, given Pk E Rdd' x2d that generates a perturbed input view and Py E RRd' x2d for the target, the self-supervised loss is defined as:
(sen(22 01-1, Pk, Pv) = [| (x zz; 0o-1) - -vvell?
(1)
where 0--1 denotes the parameters of aadppt before test-time adaptation. At each timestep I, the parameters are updated by
0. o (z-;0--1);
with learning rate Through sequential updates, fudups implicitly retains information from past visual observations, thereby capturing temporal context. Section [.6.2] we provide empirical evidence that incremental parameter updates of the value function estimator are more effective than updates performed at the trajectory level.
3.1.3 ZERO-SHOT VALUE FUNCTION ESTIMATOR
After test-time adaptation, meta-learned linear projection PQ Rd' maps the input 2, into an adaptation space Rdd". Then, this representation passed through the adaptation module fudapt- followed by regression head / that outputs predicted values of the goal-conditioned value function:
V(z g) h ff (Pq 0c))
The regression head h is two-layer multilayer perceptron (MLP), following value-function estimator architectures used in deep reinforcement learning (Espeholt al. 2018). The network is trained using normalized progress labels yt from expert demonstrations as described in Section 2.1 with supervised prediction loss 4pod defined the mean squared error between predicted values V(z g) and targets yt- At inference, the regression head h remains frozen, while the adaptation module faulapt updated online using the self-supervised objective. This enables the value function estimator to generalize in zero-shot setting without requiring task-specific demonstrations.
3.2 TRAINING PROCEDURE
We train the value function estimator with gradient- gradient-based meta-learning, optimizing a self-supervised loss Eael such that test-time adaptation improves the supervised prediction loss Éped (Sun al.) 2024). Training is performed on diverse sub- sub-trajectories selected via dissimilarity- -based sampling, which mitigates shorteut learning and encourages reliance semantic and temporal cues.
4

---

Published as a conference paper at ICLR 2026
3.2.1 META-LEARNING
During training, the adaptation module fadage is updated online each timestep using the self- supervised loss letr. Following the gradient-based meta-lcarning paradigm (Finn et al., 2017). we differentiate through these adaptation updates with respect to the supervised prediction loss Eped. This optimizes the initialization Bu of falapt, the linear projections Pk. Pv. Pq. and the regression head h. Formally, the total training loss combines the supervised prediction loss free with the self-supervised loss felts weighted by a scalar A. The learned parameters On serve as the initialization for adaptation inference time. This approach enables the value function estimator learn to adapt its internal representations at test time to improve value function estimation.
3.2.2 DISSIMILARITY- BEEEMILLARTTY-BAEDD BASED SAMPLING
Expert visual trajectories often contain consecutive frames that are highly redundant, as noted in prior work on video action recognition (Wang et al. 2018). Such redundancy can encourage the value function estimator to exploit shorteut cues (Geirhos et al.) 2020; Lee et al. (2211)) by overfitting to frequently occurring ltte-stage visual patterns. mitigate this, we propose a dissimilarity-ban sampling strategy that constructs mini-batches from the most visually diverse uub-trajectories within each trajectory. This increases intra-batch variance and acts as a form of importance sampling that favors semantic diversity.
Formally, given a sequence multimodal representations (ZI,...,TT]) we extract fixed-length sub-trajectories using a sliding window of size whr and stride s, yielding candidate set W. A diverse subset of size maximizes pairwise dissimilarity:
W
arg max UCW, [M
Σ
1 - w 12.
[w (E)
where each w is a sub-trajectory of length Wh-- Solving Eq. is intractable, requiring enumeration of all ((W) subsets. To approximate this objective efficiently, we adopt a scoring-bae based heuristic, assigning each window W score equal to its total dissimilarity with all others:
sww) Σ w|w v
W arg max
Σ s(w).
(5)
UCW. = WEl
Thus, by computing diversity score for each window and selecting the k highest-scoring windows, we obtain a heuristically diverse subset with polynomial-time complexity. The overhead is negligible compared model training, shown by complexity analysis in Appendix G We empirically validate dissimilarity-based sampling against full-trajectory sampling in the ablation study (Section I.6.Ip, showing improved ability distinguish expert from non-expert visual trajectories.
EXPERIMENTS
Our evaluation demonstrates the effectiveness of VITA across core capabilities of goal-conditioned value functions: (i) generalization in value estimation for real-world robotic manipulation under distribution shifts in task, environment, and embodiment; (ii) differentiation between expert and non-expert visual trajectories in real-world robotic manipulation tasks; (iii) effective zero-shot reward shaping for offline RL in Meta-World MT10, a simulated benchmark for multi-task robot learning.
4.1 TRAINING SETUP
We train VITA on expert visual trajectories paired with natural language task descriptions from the BridgeData V2 dataset (Walke et al. 2023), which spans wide range of manipulation tasks, environments, and robot embodiments. In particular, we use a curated subset consisting of 2,986 expert demonstrations covering pick-and-place manipulation tasks, but does not include folding, sweeping, stacking tasks. All demonstrations (WidowX 250) across 4 configurations of the ToyKitchen environment. An additional 287 expert demonstrations are held out as the in-distribution test set. referred to t k.ppnp. Appendix includes examples from the training set. We use OpenCLIP ViT-B/32 as the frozen backbone for encoding video frames and task descriptions. The training objective combines foee with beer, weighted by
collected using single robot embodiment
5

---

Published as a conference paper at ICLR 2026
(a) "Put red object into silver pot."
(b) "Fold clothes from left to right."
(c) "Sweep into pile."
(d) "Put orange object in drawer."
Figure 3: Examples of visual trajectories paired with task descriptions under different distribution
shifts. (a) In-distribution. (b, c) Environment shift. (d) Embodiment and environment shift.
Ageir = 0.5. During training, we use dissimilarity based sampling with window size War = 8, number of sub-trajectories k = 8, and stride s 1. At test time, we use one gradient step (tep 1) using A with learning rate = 0.1. Further details and hyperparameter sweeps are reported in Appendix D
4.2 BASELINES
We compare VITA against value functions based on contrastive and autoregressive VLMs. VLM-CL computes zero-shot value estimates from cosine similarity between CLIP representations of each frame and task description (Baumli et al. 2023). VLM-RM regularizes CLIP embeddings by project- ing features along the direction from generic reference prompt to the task prompt (Rocamonde et al.. 2024). CLIP-FT trains supervised regression head on frozen CLIP multimodal representations. CLIP-GRU trains Gated Recurrent Unit (GRU) (Cho et al., 2014) on frozen CLIP multimodal representations. GVL (Ma et al. (0224) is the state-of-the-art zero-shot value function that leverages autoregressive VLMs. In our experiments, GVL-OS refers to the zero-shot setting, where VLM is prompted with only the test trajectory and task description. GVL-1S corresponds to one-shot in-context setting, where a full shuffled trajectory from the training distribution, along with its progress labels, is provided Pro-latest) (Gemini Team et al.] 2024) with its proposed prompt template. We also tested Qwen- VL. 2.5 (Yang al. (2024), which failed overcome temporal bias, and GPT-4o (Hurst et al.) 2024), which refused to produce value estimates. Additional implementation details are provided in Appendix D
example. We follow GVL in using Gemini 1.5 Pro (gemini- -
4.3 EVALUATING GENERALIZATION UNDER DISTRIBUTION SHIFTS
We evaluate the ability of progress estimators generalize across novel environments, tasks, and robot embodiments using subsets of BridgeData V2 curated to introduce variation along these axes. Environment shifts involve changes in scene layout or background. For example, 1m-pnp is pick-and-place task in front of a laundry machine, while td_foldd fold, ft_fold, and rd_fold feature cloth folding on different surfaces. ms_sweep introduces a horg-oorizon sweeping task in a confined tray. Embodiment shifts are evaluated using the DeepThought robot, which differs from the training embodiment (WidowX 250) in both morphology and camera perspective. dt.tk рuΩ (pick-and- place) and dt_tk_stack (stacking) retain the in-distribution environment with new embodiment, while dt._f_sstack (stacking) and dt_rd_pnp (drawer pick-and-place involve both embodiment and environment shifts. Each dataset includes 200 expert trajectories with task descriptions, except for ms ms_sweep, which contains 100 due to its size limitation. Appendix B provides a full description of our evaluation datasets. We evaluate the performance of value function estimator using the Value Order Correlation (VOC) metric (Ma et al. 2024), which measures the alignment between the predicted progress towards task completion and the chronological order of the frames in visual

---

Published as a conference paper at ICLR 2026
Table 1: VOC scores for value function estimation under distribution shifts. ID 1n- In-Distribution, ES = Environment Shift, EM = Embodiment Shift, ES & EM = Both Shifts.
Shift
Dataset
VLM-CL
VLM-RM
CLIP-FT
GVL-0S
GVL-18
CLIP-GRU
VITA
ID
pkp
0,038
0.029
0.251
0.269
0.252
0.773
0.782
Im imp
0,017
0.033
0.149
0.305
0.272
0.676
0.725
id_fold
0,031
0.072
0.152
0.326
0.318
0.674
0.709
E8
ft.fold 0.108
0.099
0.162
0.331
0.387
0.693
0.658
rd fold 0.095
0.055
0.126
0.372
0,406
0.726
0.606
ms_sweep -0.129
-0.226
0.148
0.158
0.150
0.434
0.490
tk pnp 0.042
-0.041
0.149
0.258
0,211
0.856
0.820
RM
dt tk stack 0.035
0.046
0.099
0,254
0.277
0.667
0.708
stack 0.026
0.028
0.049
0.212
0.249
0.674
0.698
ES
EM
rd pnp
0.023
0.041
0.211
0.329
0.316
0.747
0.695
trajectory. Formally, let pr.....Pr denote the predicted progress values for trajectory of T frames, and let r, represent the temporal index. VOC is defined as the Spearman rank correlation Ps between the predicted values and the temporal indices.
In Table 1 the results show that both VITA and CLIP-GRU consistently outperform GVL-OS and GVL-15 highlighting the limitations of pre-trained autoregressive VLMs and the importance of preserving temporal order for effective progress estimation. While both demonstrate effective progress estimation, VITA outperforms CLIP-GRU on the majority of evaluation datasets (6 out of 10). Both VITA and CLIP-GRU outperform CLIP-based baselines that lack temporal modeling (CLIP-FT, VLM-CL, VLM-RM), indicating the impact of encoding temporal history on progress estimation. CLIP-FT performs comparably to GVL on the in-distribution task but fails generalize to aut-of-distribution settings. Both VLM-CL and VLM-RM perform poorly, likely due to their lack of temporal modeling. For GVL-OS and GVL-1S. S, in some cases, in-context examples improve performance, while others, they provide no benefit or even degrade performance. Both GVL methods perform well on folding tasks (td fold, ft fold, rd fold), but fail to achieve comparable performance on stacking (dt tk stack, dt ft stack) and pick-and- place (dt_tk_png) 1m_pnp) tasks, suggesting that the autoregressive VLM used in GVL may be biased toward folding-like robotic manipulations. In contrast, VITA achieves consistent performance across all task types and distribution shifts, indicating stronger generalization than in-context learning.
methods, except CLIP-FT, perform worse on the long-horizon task ms ms_sweepp but VITA achieves the highest VOC score. In particular, VITA outperforms (LU,-GRU, indicating that encoding temporal history via test-time adaptation is more effective than modeling it through re- current hidden states for long-houd morizorrzon robotic manipulation tasks. Under embodiment shifts, VITA demonstrates strong generalization. In dt. t pnp, which uses a different robot embodiment but shares the same environment and tasks the training set, both VITA and CLIP-GRU exceed their in-distribution performance, indicating that learning temporal context can effectively transfer across robot embodiments. In contrast, GVL-IS performs poorly both dt.tk-pnp and dt_tk_sta ackk, suggesting that few-shot learning fails to generalize under embodiment shifts in our evaluation setup.
4.4 DIFFERENTIATING EXPERT FROM NON-EXPERT TRAJECTORIES
Beyond generalization, we evaluate VITA's 's robustness by testing its ability to distinguish expert from suboptimal visual trajectories, assigning lower progress scores to the latter. To this end, we compare model predictions on expert and scripted (non-expert) visual trajectories collected from the same in-distribution setting of BridgeData V2. The scripted trajectories are generated in the ToyKitchen environment using heavily randomized controller (Walke et al. 2023), under the same embodiment and environmental configuration as the expert demonstrations. These include pick-and- place tasks involving object categories that may overlap with the training set, such as general objects (sc-pnp_ob) i), rigid items (sc.pnp_rob) 1), soft toys (sc.pnp.stoy) utensils (sc-pnp_ut en), and a cloth (sc pnp cloth). While these trajectories match the training distribution in task, embodiment, and environment, their behavior deviates from the smooth and monotonic progress
7

---

Published as a conference paper at ICLR 2026
Method
BinVOC
Method
IQM
95% CI
VLM-CL
0.40
VLM-CL
0.760
[0.722, 0.791]
VLM-RM
0.00
VLM-RM
0.746
[0.718. 0.771]
CLIP-FT
0.80
CLIP-FT
0.785
[0.759, 0.809]
GVL-0S
1.00
CLIP-GRU
0.777
[0.734, 0.814]
GVL-1S
1,00
META-WL
0.779
[0.750, 0.804]
CLIP-GRU
0.80
VITA
0.815
[0.785, 0.838)
VITA
1.00
(b) Offline RL on MTIO (IQM).
(a) Expert vs. Non- Mon-Expert (Bin
Table 2: (a) Success in distinguishing expert from scripted robot demonstrations, measured by average VOn across in-distribution scripted datasets. (b) Offline RL performance the Meta-World MT10 benchmark measured by interquartile mean (IQM) with 95% stratified bootstrap Cls.
typically exhibited by expert demonstrations. Appendix C includes examples from the scripted dataset. As scripted datasets are generated by randomized controller, they are not designed to reflect quantifiable levels of suboptimality. Nonetheless, an effective value function estimator should assign lower VOC scores to suboptimal trajectories relative to expert ones when evaluated in the same setting. We measure discriminative success with Bin VOC, binary evaluation metric defined as 1 [VOCexp 5OCsuhogr), which is when expert trajectories achieve higher VOC than suboptimal ones and otherwise.
The model's discriminative performance is evaluated using BINVOC, averaged over all scripted datasets. Performance on expert visual trajectories (VOCesp) is measured on the in-distribution test set tk.pnp, which features pick-and-place tasks similar to those in the scripted evaluation. VITA, GVL-OS, and GVL- achieve perfect discrimination, consistently assigning higher VOC scores to expert trajectories across all tasks. VITA outperforms CLIP- GRU, indicating that encoding temporal memory in recurrent hidden states may be more prone to overfitting to temporal shortcuts compared implicit memory via sequential test-time updates. CLIP-based baselines (VLM-CL, VLM-RM) likely underperform due to the absence of temporal modeling. CLIP-FT performs more reliably, but fails on one task, suggesting sensitivity to object distribution shifts.
4.5 ZERO- SROO-SHOT REWARD SHAPING FOR OFFLINE RL
We evaluate our value function estimator on the Pera-World MT10 benchmark (Yu et al. 2020). which consists of ten diverse robotic manipulation tasks, including pick-and-place, door opening, and pushing. This experiment designed to evaluate VITA's zero-shot reward shaping for offline RL. VITA is trained For each task in Meta- World MT10 benchmark, we generate 20 expert visual demonstrations using Meta-World's expert policies. The goal-conditioned value estimator is then used to define dense reward for each visual observation. Policies are trained with Implicit Q-Learning (IQL.) (Kostrikoy al., (022I) in the offline RL setting. We repeat expert data generation and policy training across 10 random seeds. After training, each multi-task policy is evaluated on 20 rollouts per task, and we compute the average return acToss episodes for each (task, seed) pair. We report the interquartile mean (IQM) across all pairs, together with 95% stratified bootstrap confidence intervals, following the evaluation protocol proposed by Agarwal et al.) (202T). The training details and evaluation protocol are further described in Appendix
real- world robotic data and evaluated zero-shot in this simulated setting.
Table 2 reports performance on the MT'00 benchmark for the baselines under the evaluation protocol described above. VITA achieves the highest IQM return (0.815), outperforming all CLIP-based baselines. A direct comparison with GVL is infeasible at scale, since relies on Gemini-1.5, a proprietary autoregressive VLM with prohibitively high inference cost in RL settings, while open- source alternatives did not yield reliable progress estimates in our preliminary tests. We also report the fuzzy-logie dense reward provided by Meta-World (META-WV WL). which achieves 0.779. Despite its strong performance, META- WL is outperformed by VITA, indicating that a value estimator trained on real-world data can generalize effectively simulated reward shaping.
8

---

Published as a conference paper at ICLR 2026
4.6
ABLATION STUDIES
4.6.1 EFFECT OF DISSIMILARITY- BASED SAMPLING ON DISCRIMINATIVE PERFORMANCE
We empirically validate the effectivenesss of the proposed dissimilarity- based sampling by comparing the impact of different sub-trajectory sampling strategies on VITA's ability to distinguish expert from non-expert demonstrations. In particular, full-trajectory sampling (Sun et al. 2024) computes the adaptation loss over all overlapping sub-trajectories with stride s = 1, which in our setup leads to overfitting to global temporal shortcuts. Random sampling improves diversity by selecting sub-trajectories uniformly, but lacks semantic diversity. In contrast, dissimilarity- -baed sampling explicitly promotes semantic diversity, resulting in better generalization. Our proposed method, which selects tabetrraectooies that maximize pairwise dissimilarity, outperforms both full-trajectory and random sampling in differentiating expert and non-exper visual trajectories. For further details, please refer to Appendix E.
4.6.2 EFFECT OF TEMPORAL MEMORY ON VALUE FUNCTION ESTIMATION
We analyze how different test-time adaptation strategies for incorporating temporal context impact value function estimation. VITA performs step-by-step adaptation without reset (implicit memory). incrementally updating parameters from ---1- This allows the value function estimator to encode trajectory history directly into its parameters, capturing both temporal order and accumulated temporal context. We compare VITA with three alternatives: (i) trajectory-level adaptation (TTT-TR), which applies a single gradient update computed over the full trajectory. This produces batched update that averages the loss across all frames, thereby discarding temporal order; (ii) memoryless adaptation (TTT-RS), which resets the adaptation module to 0u at every step. By adapting only to the current visual observation, this approach prevents any temporal information from being carried forward; (iii) explicit memory (TTT-EX), which also resets the adaptation module every step but updates using local window of recent frames matching the window size used during meta-training (wr = 8). Across all datasets, VITA consistently outperforms trajectory-level, memoryless, and explicit-memory methods, demonstrating the effectiveness of incrementally accumulating temporal context for value function estimation. We provide the full analysis in Appendix F.2
RELATED WORK
5.1 VLMS AS VALUE FUNCTION ESTIMATORS
Contrastive VLMs have been applied as zero-shot reward models (Baumli et al. 2023 Rocamonde et al.) 2024) and goal-conditioned value functions (Ma et al. 2023b 2024). using frame-level
similarity scores but lacking temporal modeling. Some works fine-tune CLIP on domain-specific
video datasets (Fan et al.) 2022 Jiang et al.] (0224) to generate dense rewards, requiring supervision that limits their zero-shot applicability. RoboCLIP (Sontakke et al. (023) enables one-shot reward specification without tuningg but yields sparse rewards and still requires demonstrations. Large-
scale multimodal pre-training approaches (Alakuijala et al. 2024 Li et al. 2024 Ma et al., 2023b) learn canguage-conditionee value functions by relying on scale for generalization. ReWiND (Zhang et al. 2025b) trains a cross-modal sequential transformer on augmented large-scale data with video rewinding, combined with few in-domain demonstrations to improve generalization. In contrast, VITA adapts a pre-trained contrastive Vasionnlanggage encoder at inference, capturing both temporal and semantic context through test-time adaptation, without requiring large-scale multimodal pre- training or additional domain-specine demonstrations for generalization. VITA could be applied to any pre-trained lanion-language representation that encodes universal value function to enhance its generalization and temporal reasoning via meta-learned test-time adaptation. Autoregressive VLMs (Alayrac et al. 2022) leverage in-context learning over trajectories and have been applied to success detection (Du et al. 2023) and goal-conditioned value functions (Ma et al. 2024). GVL (Ma) jet al. 2024) mitigates monotonic bias from ordered pre-training data by shuffling frames at inference. but this discards temporal order, which is essential for temporal reasoning and for distinguishing visually similar yet temporally distinct states. Our approach instead preserves chronological order while capturing temporal context through test-time adaptation, avoiding such shorteuts.

---

Published as a conference paper at ICLR 2026
5.2 TEST-TIME ADAPTATION
Test-time adaptation methods provide parameter-effiient solutions for adapting VLMs without the need for full fine-tuning. Prompt-based methods optimize model parameters at test time based on the task context, although their underlying mechanisms are opaque (Zhou et al. 2022 Khattak et al. 2023 et al. 2023a). Lim et al. (2025) suggested that CLIP representations could support test-time adaptation, based on the observation that CLIP encodes human-iilerpretable concepts discovered via mechanistic interpretability (Bricken et al. 2023). CLIP- similarity has also been used as reward signal, with parameters optimized test time via reinforcement learning (Zhao et al. 2024). Another approach is test-time training (Sun et al. 2020), where an adaptation module is updated at each timestep via a self-supervised loss. Unlike meta-reiinforemmett learning methods (Finn et al.) 2017 Bauer et al. 2023), which require task-specific adaptation episodes during training, test-time training enables direct online adaptation during inference, without the need for task labels.
6 DISCUSSION, LIMITATIONS, AND FUTURE WORK
6.1 DISCUSSION
We show that test-time adaptation enables value functions for robotic manipulation to generalize across distribution shifts in task, environment, and embodiment. In addition, VITA can distinguish expert from non-expert visual trajectories and perform zero-shot reward shaping, enabling downstream applications in RL and imitation learning. By updating sequentially over trajectory, VITA encodes history into its parameters, capturing temporal and semantic context more effectively than CLIP- based baselines and in-context learning with autoregressive VLMs. VITA outperforms CLIP- GRU discrimination and offline RI, indicating that implicit memory via sequential
on expert-non-ewt test-time updates generalizes more effectively than explicitly encoding temporal history in recurrent hidden states for zero-shot value function estimation. Pre-trained autoregressive VLMs (GVL), despite their generalization capabilities, are not explicitly trained for progress estimation, which requires temporal reasoning over visual trajectories. In our setup, the inference-time overhead of per-timestep adaptation is negligible, as only a lightweight adaptation module is updated at test time, which does not affect real-time applicability.
6.2 LIMITATIONS AND FUTURE WORK
Test-time adaptation improves generalization of zero-shot progress estimation across unseen tasks and environments in our experiments, but it can still face challenges in settings with high execu- tion variability or extended durations. Although the adaptation overhead is negligible due to the lightweight adaptation module, updating a value function estimator at every timestep may be poten- tially unsafe during deployment, limiting applicability in real-time scenarios. In future work, we plan to explore alternative approaches for test-time adaptation of vision-language models, particularly in the context of training agents within world models. In addition, applying VITA to real-time closed-loop control settings and complex RL environments remains promising direction for future work. VITA could enhance generalization and temporal reasoning for any domain-specific pre-trained multimodal representation via learnee test-time adaptation, as it is agnostic the underlying vision- language encoder. While we provide empirical evidence that dissimilarity-basce sampling mitigates shortcut learning, theoretical analysis of how divere sampling influences shorteut learning remains a promising direction. This lies beyond the scope of our core contribution, which introduces value function estimator that meta-learns a test-time adaptation mechanism improve zero-shot performance and temporal reasoning.
ACKNOWLEDGMENTS
We thank Daniel Furelos-Ba -lanno, Roko Parac, and Frederik Kelbel for their helpful comments on an earlier draft. We also thank Vassilis Digalakis for valuable discussions. This work was supported by UKRI (EP/Y037111/I) as part of the ProSafe project (EU Horizon 2020, MSCA, grant no. 101 1 19358).
10

---

Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Krzysztof Choromanski, Tianli Ding. Danny Driess, Kumar Avinava Dubey, Chelsea Finn, Peter R. Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan C. Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Sergey Levine, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael S. Ryoo, Grecia Salazar, Pannag R. Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Ho Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Ted Xiao, Tianhe Yu, and Brianna Zitkovich. models transfer web knowledge to robotic control. In Conference on Published as a conference paper at ICLR 2026
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical al. arXiv preprint arXiv:2501.033 2025.
Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare. Deep reinforcement learning at the edge of the statistical precipice. In Advances in Neural Information Processing Systems, 2021.
Minttu Alakuijala, Reginald McLean, Isaac Woungang, Nariman Farsad, Samuel Kaski, Pekka Marttinen, and Kai Yuan. Video-language critic: Transferable reward functions for language- conditioned robotics. arXiv preprint arXiv:2405. 19988, 2024.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei. Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. Flamingo: a visual language model for few-shot learning. arXiv preprint arXiv:2204. 14/98, 2022.
Bowen Baker, Ilge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. Advances in Neural Information Processing Systems, 2022.
Jakob Bauer, Kate Baumli, Feryal M. Behbahani, Avishkar Bhoopchand, Nathalie
Michael Chang, Natalic Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Thossshomsonn, Hannah Openshaw. Jack Parker-Holder, Shreya Pathak, Nicolas Perez Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Satinder Singh, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, and Lei M. Zhang. Human-timescale adaptation in an open-ended task space. In International Conference on Machine Learning, 2023.
Kate Baumli, Satinder Baveja, Feryal M. P. Behbahani, Harris Chan, Gheorghe Comanici, Sebastian Flennerhag. Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, Clare Lyle, Hussain Masoom, Kay McKinney. Volodymyr Mnih, Alexander Neitz, Fabio Pardo, Jack Parker- Holder, John Quan, Tim Rocktäschel, Himanshu Sahni, Tom Schaul, Yannick Schroecker, Stephen Spencer, Richie Steigerwald, Luyu Wang, and Lei Zhang. Vision-language models as a source of rewards. arXiv preprint arXiv:2312.09113 2023.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph. Zac Hatfield-Dodde Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Rt-2:
Robot Learning. 2023.
Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Maria Elis- abeth Bechtle, Feryal Behbahani, Stephanic C.Y. Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando De Freitas,
11

---

Published as a conference paper at ICLR 2026
Satinder Singh, and Tim Rocktäschel. Genie: Generative interactive environments. In International Conference on Machine Learning, 2024.
Annie S Chen, Suraj Nair, and Chelsea Finn. Learning generalizable robotic reward functions from in-the-wild human videos. arXiv preprint arXiv:2103.16817, 2021.
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv: /406. /078, 2014.
Nitish Dashora, Dibya Ghosh, and Sergey Levine. Viva: Video-trained value functions for guiding online from diverse data, arXiv preprint arXiv:2503.18210 2025.
Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, and Serkan Cabi. Vision-langusee models success detectors. In Conference Lifelong Learning Agents, 2023.
Ashley Edwards, Himanshu Sahni, Yannick Schroecker, and Charles Isbell. Imitating latent policies from observation. In International Conference on Machine Learning, 2019.
Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, lain Dunning, Shane Legg, and Koray Kavukcuoglu. IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures. In International Conference on Machine Learning, 2018.
Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang. Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. In Advances in Neural Information Processing Systems, 2022.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, 2017.
Robert Geirhos, Jömn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2020.
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 2024.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016.
David Ha and Jürgen Schmidhuber. World models. In Advances in Neural Information Processing Systems, 2018
Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. International Conference on Learning Representations, 2021.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Advances in Neural Information Processing Systems, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 1997.
Anthony Hu, Lloyd Russell, Hudson Yeo. Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton. and Gianluca Corrado. GAIA-I: A generative world model for autonomous driving. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
Edward Hughes, Michael D. Dennis, Jack Holder, Feryal M. P. Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul, and Tim Rocktäschel. Position: Open-endedness is essential for artificial superhuman intelligence. In International Conference Machine Learning, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
12

---

Published as a conference paper at ICLR 2026
Haobin Jiang, Jumpeng Yue, Hao Luo, Ziluo Ding, and Zongqing Lu. Reinforcement learning friendly vision-language model for minecraft. In European Conference on Computer Vision, 2024.
Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Paul Foster, Pamnag R. Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model. In Conference Robot Learning. 2024.
Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06166 2021.
Youngwoon Lee, Andrew Szot, Shao-Hua Sun, and Joseph J. Lim. Generalizable imitation learning from observation via inferring goal proximity. In Advances in Neural Information Processing Systems, 2021a.
Youngwoon Lee, Andrew Szot, Shao-Hus Sun, and Joseph J. Lim. Generalizable imitation learning from observation via inferring goal proximity. In Advances in Neural Information Processing Systems, 2021b.
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement Icarning Tutorial, review, and perspectives open problems. arXiv preprint (tx.ix:2005006644 2020.
Jianxiong Li, Jinliang Zheng, Yinan Zheng, Liyuan Mao, Xiao Hu, Si Cheng, Haoyi Niu, Jihao Liu, Yu Liu, Jingjing Liu, Ya-Qin Zhang, and Xianyuan Zhan. Decisionnce: Embodied multimodal representations via implicit preference learning. In International Conference on Machine Learning. 2024.
Hyesu Lim, Jinho Choi, Jacgul Choo, and Steffen Schneider. Sparse autoencoders reveal selec- tive remapping of visual concepts during adaptation. In International Conference Learning Representations, 2025.
Ziyi Lin, Honglie Liu, Andrew Zisserman, and Andrea Vedaldi. Frozen clip models are efficient video learners. In European Conference on Computer Vision, 2022.
Xudong Ma, Chen Liang, Zhecan Wang, Chuang Gan, and Zhiting Li. Swapprompt: Test-time prompt adaptation for vision-language models. In Advances in Neural Information Processing Systems, 2023a.
Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and representation via value-implic gre-training. arXiv preprint wxXiv22100000300 2022.
Yecheng Jason Ma. Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. LIV: Language-iman -mage representations and rewards for robotic control. In International Conference on Machine Learning, 2023b.
Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani. Peng Xu, Danny Driess, Ted Xiao, Jonathan Tompson, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, and Fei Xia. Vision language models are in-context value learners. In International Conference on Learning Representations, 2024.
Yutaka Matsuo, Yann LeCun, Maneesh Sahani, Doina Precup, David Silver, Masashi Sugiyama, Eiji Uchibe, and Jun Morimoto. Deep learning, reinforcement learning, and world models. Neural Networks, 2022.
Viorica Päträucean, Yusuf Aytar, Roozbeh Mottaghi, Oleh Rybkin, Mario Lucic, Carl Vondrick. William T. Freeman, and Andrew Zisserman. Perception test: diagnostic benchmark for multimodal video models. In Advances in Neural Information Processing Systems, 2023.
13

---

Published as a conference paper at ICLR 2026
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley D. Edwards, Nicolas Manfred Otto Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals. Mahyar Bordbar, and Nando de Freitas. A generalist agent. arXiv preprint arXiv:2205.06/75. 2022.
Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision- language models are zero-shot reward models for reinforcement learning. In International Confer- ence on Learning Representations, 2024.
Dominik Schmidt and Minqi Jiang. Learning to act without actions. In International Conference on Learning Representations, 2024.
David Silver and Richard S. Sutton. Welcome to the era of experience. Technical report, Google AI, 2025.
SIMA Team, Maria Abi Raad, Arun Ahuja, Catarina Barros, Frederic Besse, Andrew Bolt, Adrian Bolton, Bethanie Brownfield, Gavin Buttimore, Max Cant, et al. Scaling instructable agents across many simulated worlds. arXiv preprint arXiv:2404. 10179, 2024.
Sumedh Sontakke, Jesse Zhang, Sébastien M. R. Arnold, Karl Pertsch, Erdem Biyik, Dorsa Sadigh, Chelsea Finn, and Laurent Itti. Roboclip: One demonstration is enough to learn robot policies. In Advances in Neural Information Processing Systems, 2023.
Yu Sun. Xiaolong Wang. Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test- time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning. Proceedings of Machine Learning Research. PMLR, 2020.
Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. Learning to (learn at test time): Rons with expressive hidden states. arXiv preprint 7XXv:2447.004200. 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
Homer Rich Walke, Kevin Black, Tony Z. Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, Abraham Lee, Kuan Fang, Chelsea Finn, and Sergey Levine. Bridgedata v2: A dataset for robot learning scale. In Conference on Robol Learning, 2023.
Limin Wang, Yuanjun Xiong, Dahua Lin, and Lue Van Gool. Temporal segment networks: Towards good practices for deep action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
Renhao Wang. Yu Sun, Yossi Gandelsman, Xinlei Chen, Alexci A. Efros, and Xiaolong Wang. Test-time training video streams. arXiv preprint arXiv:2307.05011 2023.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men. Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
Ze Yang, Yiming Zhang, Jenyy Yu, Junru Cai. and Ming- Hiugg Yang. UniSim: A neural closed- sensor simulator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
14

---

Published as a conference paper at ICLR 2026
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference Robot Learning. 2020.
Chuheng Zhang, Tim Pearce, Pushi Zhang, Kaixin Wang, Xiaoyu Chen, Wei Shen, Li Zhao, and Jiang Bian. Whaa do latent action models actually learn? arXiv preprini arXiv:2506. 15691, 2025a.
Jiahui Zhang. Yusen Luo, Abrar Anwar, Sumedh Anand Sontakke, Joseph J Lim, Jesse Thomason, Erdem Biyik, and Jesse Zhang. RewiND: sanguaggeguuidenn rewards teach robot policies without new demonstrations. In Conference on Robot Learning, 2025b.
Shuyuan Zhao, Xin Wang, Linchao Zhu, and Yi Yang. Test-time adaptation with clip reward for zero-shot generalization in vision-language models. arXiv preprint XxXx:2401.12355 2024.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, 2022.
15

---

Published as a conference paper at ICLR 2026
A
TRAINING DATASET
(a) "Put red object into silver pot."
(b) "Take silver bowl lid from table."
(c) "Place yellow object inside pot."
(d) "Move red object from pot to left burner."
Figure 4: Each subfigure shows the start and end frames from an expert demonstration used for training, along with its natural language task description. Demonstrations are collected across four distinct ToyKithhen environments.
B EVALUATION DATASETS
Table Dataset descriptions with task type, environment, and embodiment.
Dataset
Task Type
Environment
Embodiment
tk.pnp
pick-and-place
toy kitchen
WidowX 250
1m.pnp td_fold
pick-and-place
laundry machine
WidowX 250
fold cloth
tabletop (dark wood)
WidowX 250
ft.fold
fold cloth
folding table
WidowX 250
rd_fold
fold cloth
robot desk
WidowX 250
ms_sweep
sweep
folding table (tray)
WidowX 250
dt tk pnp dt.tk_stack
pick-and-place stack blocks
toy kitchen
DeepThouugh
toy kitchen
ThepThh
dt.ft.stack dt dt.rd.pnp pnp
stack blocks
folding table robot desk (drawer)
DeepThought
pick-and-place
DeepThoough
16

---

Published as a conference paper at ICLR 2026
(a) at.tk.ns tk.ns: "Move yellow object from burner center."
(b) dt .t.kktkkk: "Place green cube on top of arch."
(c) dt. st.ckckk:: "Move egg table."
(d) dt dt_rd pnp: atpr "Put orange object drawer."
Figure 5: Each subfigure shows the start and end frames from an evaluation trajectory under embodi- ment shift, along with its natural language task description. The top row depicts tasks in the same environment (ToyKitchen) using a different robot (DeepTThought), while the bottom row includes tasks that also involve new environments.
RIGE
u
(a) 1m-pnp: "Place blue cloth inside washer."
(b) foldold: "Fold cloth bottom right to top left."
(c) Êt_Ecppp: "Fold clothes from to right."
(d) rd.fola: "Fold cloth bottom right to top left."
(c) ms. ft..wwep: "Sweep into pile."
Figure 6: Each subfigure shows the start and end frames from an evaluation trajectory under environ- ment shift, along with its natural language task description. All tasks are performed using the same robot embodiment across visually and structurally different environments.
17

---

Published as a conference paper at ICLR 2026
C
SCRIPTED DATASET
(a) sc.pnp_obij "Pick up an object and place it."
(b) sc.pnp_robj: "Pick up rigid object and place it."
(c) sc-pnp.stoy: "Pick up soft object and place it."
(d) sc-pnp-uten: "Pick up kitchen utensil and place it."
(c) sc. sc.pnp_cloth: "Move the cloth."
Figure Each subfigure shows the start and end frames from scripted evaluation trajectory collected in the ToyKitchen enviromment using the WidowX 250 robot. All tasks involve pick-and- place behavior with varying object categories. We use generic task descriptions due the lack of publicly available annotations.
D MODEL ARCHITECTURE
D. MODEL. AND TRAINING DETAILS
We the OpenCLIP ViT-B/32 encoder pre-trained the OpenAl dataset frozen backbone. Each visual observation and task description is encoded using the CLIP vision and text encoders to produce their representation, respectively. We opted for a joint (coneatenation-based) representation over an element-wise product based on improved validation performance. The adaptation module falupt is two-layer residual MLP with GELU activation and projection dimension d' 64. The model is trained using the AdamW optimizer with learning rate of 10 4, weight decay of 1 x 10-4 and cosine learning rate schedule with 10% warmup. We use a batch size of 32 and pad all trajectories a maximum length of 120 frames (matching the longest sequence in the training set). The weighting coefficient A of the self-supervise loss in the total training objective is set to 0.5, selected based on validation performance. We train for epochs, as validation VOC typically plateaus early, with extended training providing no further improvement. For dissimilarity- -ased sampling, we set the stride to s the sub-trajectory length to wi = 8, and the number of selected windows to k 8, unless otherwise specified. All experiments were NVIDIA RTX 6000 Ada Generation GPUs.
D.2 TEST-TIME TRAINING HYPERPARAMETERR
At inference time, we adapt only the temporal adaptation module falams using the same projec- tion dimension (d' 64) in training. We perform single gradient step (lep 1) with
18

---

Published as a conference paper at ICLR 2026
Table 4: Effect of test-time learning rate 77 and number of adaptation steps top on VITA performance on the validation set measured by VOC.
7
lep
tep
lpp 10
0.001
0.225
0.345
0.453
0.01
0.454
0.744
0.781
0.05
0.746
0.670
0.515
0.1
0.783
0.512
0.369
1.0
0.299
0.183
0.181
5.0
0.169
0.182
0.157
learning rate 77 0.1. This configuration was selected from a sweep over learning rates (5.0, 1.0, 0.1, 0.05. 0.01, 0.001} and adaptation steps {1,5, 5, 10), based on validation performance reported in Table
The results show that both (i) a single update with a larger learning rate
(n 0.1, tep 1) and (ii) multiple updates with smaller learning rate (17 = 0.01, tep 10) achieve comparable validation performance under test-time training. We select the former because requires fewer adaptation steps while achieving slightly better results.
D.3 BASELINES
VITA uses lep = 1, projection dimension d' 64, and learning rates 7i of 0.1, selected via hyperparameter tuning descrihed previously. CLIP-FT follows VITA's architecture, but excludes the adaptation module and self-supervised loss. In particular, CLIP-FT uses frozen CLIP encoder, followed by linear projection and two-layer MLP to predict task progress. The model trained with standard supervised regression, without meta-learning or test-time adaptation. To increase expressivity, it uses an 8x larger projection matrix and 10x more training steps than our method (Lin et al.] 2022). For VLM-RM, we use a baseline prompt that describes the environment. CLIP-GIA GRU follows VITA's architecture (frozen CLIP encoder and regression head h), but replaces test-time adaptation module Sudape with a Gated Recurrent Unit (GRU) module that explicitly encodes temporal history through recurrent hidden states. The GRU module has a single layer with hidden size 64, and its final hidden state is passed to the regression head h. For bothGVL-OS and GVL-IS, (Gemini Team et al.) 2024).
use the latest version of Gemini 1.5 Pro, Pro..ennii--.---pr---aa- whereas the original GVL implementation used earlier release, gemini-1.5-pp
We also
evaluated GVL using the open-source autoregressive VLM Qwen-VL 2.5 (Yang et al. 2024).
which failed overcome temporal bias despite frame shuffling, while GPT-4o (Hurst 2024)
consistently declined to produce scalar progress estimates in our setup. All evaluations followed the prompt template introduced in the original GVL work (Ma et 2024).
E
OFFLINE RL SETUP
E. TRAINING CONFIGURATION
We Implicit Q-Learning (IQL) with the following settings, kept fixed across all tasks. Expec- tile regression is set to T = 0.7, with advantage weighting temperature 3.0 and clipping thresh- old 100.0. The actor and critic use learning rates 10 and 10-4, respectively, with batch size 256. Policies are trained for 100,000 gradient steps. with evaluation every 10,000 steps on 20 rollouts per task (horizon 150). Implementation follows d3rlpy, with configura- tion: expectile=0- weight_temp=t 0,, max.weight=100. 0, actor actor_1r=le44, critic.lr=3e-0 batch.size=255.
E.2 EVALUATION PROTOCOL
Following Agarwal et al. (Agarwal et al. 2021), report the interquartile mean (IQM) of returns across all task seed pairs. The IQM computes the mean of the middle 50% scores, which reduces sensitivity outliers compared to the mean median. For statistical robustness, estimate
19

---

Published as a conference paper at ICLR 2026
95% confidence intervals using stratified bootstrap with 10,000 resamples, stratified by task. This procedure is applied consistently across all methods in Meta- World MT10 experiments.
F ABLATION STUDIES
F.1
EFFECT OF DISSIMILARITY-BASE BASED SAMPLING ON DISCRIMINATIVE PERFORMANCE
Table 5: Disc@ VOC results across scripted datasets under different training-time sampling strategies for test-time adaptation. A checkmark indicates successful discrimination between expert and suboptimal trajectories. Sampling methods FT (full-trajectory), Rand (random sampling), and Diss (dissimilarity-base) sampling) are parameterized by (Wn.. k).
Dataset
FT (8,-)
Rand (8,8) Diss (8,4)
Diss (4,8) Diss (8,8)
sc. sc.pnp.obj
sc.pnp_robj
sc.pnp_stoy
sc.pnp.uten sc sc.pnp.cloth
Avg. @isc@VOC 0.20 0.8
1.0
0.60
1.0
We empirically validate the effectiveness of the proposed dissimilarity- -based sampling by comparing the impact different sub-trajectory sampling strategies. In particular, we evaluate how different sub-trajectory sampling strategies during training impact the ability of model to discriminate expert from suboptimal trajectories. In Table 5 we compute Disc@ VOC for 5 scripted datasets, capturing whether given model assigns a higher average VOC expert trajectories (t.-ppp) than to each evaluation dataset. We compare three strategies: (1) full-trajectory sampling applies adaptation over all overlapping sub-trajectories of length Wh, using stride s, following prior work (Sun) et al., 2024;; ; (2) random sampling uniformly samples & -b-ttrajectoris of length Whr per video; and (3) our proposed dissimilarity-bon -ased sampling constructs a candidate set of tub-trajectories of length Whr using stride and selects k sub-trajectories that maximize pairwise dissimilarity in the representation space.
Our results show that dissimilarity- -based sampling with wa = 8 and k == 8 yields perfect discrimina- tive performance. Sampling fewer windows (k = 4) of size Wh = 8 lcads to the same performance, while reducing sub-trajectory length to Whr 4 reduces effectiveness. The full-trajectory baseline performs the worst, confirming that adapting over all consecutive windows can lead to overfitting to global temporal cues. Random sampling outperforms tulletrajectory sampling but remains less reli- able than dissimilarity-be bbased selection. These findings highlight the importance of both sub-trajectory diversity and semantic locality in enabling discriminative test-time adaptation.
F.2 EFFECT OF TEMPORAL MEMORY ON VALUE FUNCTION ESTIMATION
We evaluate four adaptation strategies: (1) TTT-TR, which performs a single gradient update computed over the full trajectory (offline). TTT-TR performs batched update that averages the loss across all frames, thereby discarding temporal order; (2) TTT-RS, which resets the adaptation module every step and updates using only the current visual observation (memoryless). TTT-RS prevents any temporal information from being carried forward: (3) TTT-EX, which resets the adaptation module at every step and updates using a local window of recent frames matching the window size used during meta-training (Wr = 8). TTT-EX averages gradients over this window, discarding the temporal ordering within it; (4) VITA (OURS), which updates the adaptation module incrementally at every timestep (implicit memory). VITA forms an implicit memory through the sequential accumulation of temporal history in the adapter parameters. We performed the same test-time hyperparameter sweeps for all variants, described in Appendix D. We found that VITA, TTT-RS, and TTT-TR achieve optimal performance with a learning rate of 0.1 and a single adaptation step. whereas TTT-EX performs best with leaming rate of 1. and one adaptation step.
20

---

Published as a conference paper at ICLR 2026
Table 6: Validation VOC scores for progress estimation under distribution shifts, ID Di-DDistributinn
ES = Environment Shift, EM Embodiment Shift, ES+EM Environment and Embodiment Shift.
Shift
Dataset
TTT-TR
TTT-RS
TTT-EX
VITA
ID
tk-pnp
0.1917
0.1918
0.1957
0.7822
Im_pnp
0.1843
0.1854
0.1826
0.7246
td_fold
0.1402
0.1393
0.1322
0.7085
ES
folf
0.1302
0.1311
0.1683
0.6583
folfold 0.1161
0.1172
0.1154
0.6056
ms_sweep -0,0225
-0.0191
0.0102
0.4898
dt_rk_pnp 0.2123
0.2118
0.2069
0.8203
EM
dt.tk_stack 0.0833
0.0830
0.0747
0.7081
dt_ft_stack 0.0558
0.0537
0.0439
0.6979
ES+EM
dtprd_pup 0.1665
0.1660
0.1559
0.6951
The results in Table 6 show that TTT-TR. TTT-RS, and TTT-EX yield comparable performance. This indicates that performing batched whether over short explicit window (TTT-EX) or over the full trajectory (TTI-IRR)-eerfommss contrast, VITA consistently outperforms all three variants, underscoring the importance of sequential, per-timestep updates for effective value function estimation in real-world robotic manipulation tasks. In implicit memory (VITA), updating over a local window of frames would cause the most recent observations in that window to be incorporated through the accumulated parameters. In addition, updating less frequently sonnenuuuuttalll) would cause information loss because intermediate frames omitted. Therefore, the implicit-memogy test-time training paradigm requires sequential, per-timestep updates without reset.
performs similarly to the memoryless baseline (TTT-RS). In
exiieeexplicityy through the window and implicitly
G COMPLEXITY ANALYSIS OF DISSIMILARIT-BBAEED SAMPLING
Let W denote the candidate set of size N = [W]. To approximate diverse subset selection without the exponential cost of enumerating all (%) subsets as described in Eq, 4 we adopt a scoring-based heuristic that assigns each candidate window diversity score equal to the sum of its pairwise distances to all others (i.e., the row sum of the distance matrix). Let the number of candidate windows be No T Wi + each represented by flattened feature vector of dimension m Wh : where d is the dimension of each multimodal representation 2;- We compute the full pairwise distance matrix D Ne on the GPU using batched cdist, which requires O uhr d) operations.
In our experiments, training trajectories are short (mean length 69, max length 120). With batch size 32, feature dimension d = 1024 (CLIP-based multimodal representation), and window length Wh 8, the worst-case computational cost is:
32 - N2 Wh d = 32 1142 1,22. 8 1024 ~ 340 MFLOPs.
This overhead negligible compared the cost of single forward and backward pass through our model, while promoting the selection of diverse windows.
21


Reviews:
[{"review_id": "1D361ANVu7", "reviewer": "Reviewer_M6nz", "summary": "This paper introduces VITA, a zero-shot goal-conditioned value function learning framework designed to enhance generalization and temporal reasoning through test-time adaptation (TTT).\nThe core idea is to integrate a lightweight adaptation module that is updated online via a meta-learned self-supervised loss. Each adaptation step refines value estimation, enabling the model to implicitly capture temporal dependencies within its parameters. To avoid shortcut learning, the authors propose a dissimilarity-based sampling strategy that encourages semantic diversity during training.\n\nEmpirically, VITA achieves state-of-the-art performance on real-world robotic manipulation tasks, outperforming both autoregressive zero-shot VLMs (e.g., GVL) and CLIP-based baselines. Moreover, on the Meta-World MT10 benchmark, VITA demonstrates effective zero-shot reward shaping, even surpassing the simulator’s fuzzy-logic dense rewards.", "questions": "1. TTT Variants Explanation: In Table 5, VITA outperforms both TTT-TR and TTT-RS, despite all models sharing the same network architecture. This observation raises several questions:\n1.1 Could the authors clarify why the single-trajectory update variant (TTT-TR) performs worse than the baseline?\n1.2 Were all training hyperparameters (e.g., learning rate, adaptation steps, λ₍self₎) kept consistent across these variants?\n1.3 Have the authors explored different adaptation frequencies (e.g., updating every n steps) to evaluate how temporal update intervals influence performance?\n\n2. Online RL Integration: How would VITA perform in an online reinforcement learning setting, where the agent interacts with the environment continuously and receives real-time feedback? Discussing or experimenting with this setup could help demonstrate the method’s robustness and adaptability beyond offline contexts.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes an implicit memory formulation that effectively incorporates temporal modeling into value estimation. In addition, the proposed dissimilarity-based sampling method mitigates overfitting to chronological shortcuts—a nontrivial improvement over prior work.\n\n2. Experimental results demonstrate that ViTA consistently and substantially outperforms existing baselines across diverse tasks and domains.", "weaknesses": "1. While the proposed dissimilarity-based sampling method is conceptually interesting, the paper lacks a formal theoretical explanation or empirical justification for why this approach effectively mitigates shortcut learning. Providing analytical or visual evidence would strengthen the technical soundness of the claim.\n\n2. Key hyperparameters (e.g., learning rate, adaptation steps) are only mentioned as “selected from a sweep,” leaving uncertainty about model sensitivity.\n\n3. The paper does not clarify why variants such as TTT-TR (same structure, fewer updates) perform worse than baselines.\n\n4. Some formatting inconsistencies remain (e.g., inconsistent capitalization of Lself).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dxCwOCuiyv", "reviewer": "Reviewer_DuEc", "summary": "This paper proposes VITA, a novel method for zero-shot goal-conditioned value function learning through test-time adaptation of vision-language models. The key contributions include: (1) A test-time adaptation framework that enhances both generalization and temporal reasoning capabilities of contrastive VLMs; (2) A dissimilarity-based sampling strategy to mitigate shortcut learning; (3) Comprehensive evaluation showing state-of-the-art performance on real-world robotic manipulation tasks under various distribution shifts, and successful application to offline RL reward shaping in simulated environments.", "questions": "The authors claim that their method has the advantage of \"implicit memory through parameter updates.\" Could we propose the most direct comparative baseline, which would be using an explicitly memory-equipped network (e.g., RNN, Transformers) to explicitly form parameterized memory?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The integration of test-time adaptation for temporal reasoning in zero-shot value functions is novel and creative;\n2. Extensive experiments across real-world and simulated environments demonstrate robust performance;\n3. Addresses important limitations of current VLM-based value functions and enables better generalization without task-specific fine-tuning.", "weaknesses": "1. The computational overhead of per-timestep adaptation may limit real-time applicability in some robotic systems;\n2. Lack of comparison with model baselines possessing explicit memory capabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g77iI64JXD", "reviewer": "Reviewer_ECyi", "summary": "This work proposes using Test-time adaptation modules to improve language-goal conditioned value estimation. Experiments demonstrate that it performs better than GVL (a prompting method) and other VLM based CLIP reward functions.", "questions": "* Just to clarify, TTT is run on each latent observation (concat of visual and lang embed from clip) within a window, and the value function is predicted after the final observation post adaptation.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* the paper does a good job explaining TTT\n* the idea is rather intuitive -- by gaining more context about the current setting we can produce a better value estimate.\n* The paper evaluates both VOC and also running offline RL using the value estimates as reward.\n* the data weighting scheme for diverse sampling makes sense.\n* the paper performs well on the bridge data setting in comparison to GVL.", "weaknesses": "* The paper could benefit substantially from a more detailed figure showing the overall architecture (concatenating clip latents), then how each of the projection matrices are used for TTT, and then how the final observation in the window is used for estimating the value function. I think this would help clarify the total overall flow.\n* The paper does not consider other value learning baselines like LIV (ma et al.) which consider langauge and goal representations are also based on CLIP. Generally the paper seems to lack a number of baselines which actually train on the target data like the proposed TTT method. In this regard, LIV seems like a rather natural baseline.\n* MetaWorld experiments where the scripted expert demonstrations are used are generally weak as most tasks in the benchmark can be solved by BC on ~10-20 scripted demos! So I'm not sure how significant the policy learning results are (it might be good to include a BC baseline, since IQL is basically weighting the data by the value fn).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)