MetaVLA: Unified Meta Co-Training for Efficient Embodied Adaptation
Authors: Chen Li
Zhantao Yang
Han Zhang
Fangyi Chen
Chenchen Zhu
Anudeepsekhar Bolimera
Marios Savvides
Venue: ICLR 2026 Poster (2026)
Keywords: Vision–Language–Action models
Efficient Robot Reasoning
Generalization
Primary area: foundation or frontier models, including LLMs

Abstract: Vision–Language–Action (VLA) models show promise in embodied reasoning, yet remain far from true generalists—they often require task-specific fine-tuning, incur high compute costs, and generalize poorly to unseen tasks. We propose MetaVLA, a unified, backbone-agnostic post-training framework for efficient and scalable alignment. MetaVLA introduces Context-Aware Meta Co-Training, which consolidates diverse target tasks into a single fine-tuning stage while leveraging structurally diverse auxiliary tasks to improve in-domain generalization. Unlike naive multi-task SFT, MetaVLA integrates a lightweight meta-learning mechanism—derived from Attentive Neural Processes—to enable rapid adaptation from diverse contexts with minimal architectural change or inference overhead. On the LIBERO benchmark, MetaVLA with six auxiliary tasks outperforms OpenVLA by up to 8.0\% on long-horizon tasks, reduces training steps from 240K to 75K, and cuts GPU time by 76%. These results show that scalable, low-resource post-training is achievable—paving the way toward general-purpose embodied agents. Code will be available.

Published as a conference paper at ICLR 2026
METAVLA: UNIFIED META CO-TRAINING FOR EFFI- CIENT EMBODIED ADAPTATION
Chen Li
Zhantao Yang
Carnegie Mellon University chen @andrew. cmu.edu
Carnegie Mellon University
zhantaoy@andrew cmu edu
Han Zhang
Fangyi Chen Carnegie Mellon University
Chenchen Zhu
Carnegie Mellon University hanz3 cmu. edu
Meta Reality Labs, USA chenchenz@meta com
fangyiceandrew. emu, edu
Anudeepsekhar Bolimera
Marios Saavides
Carnegie Mellon University abolimes andrew. cmu. edu
Carnegie Mellon University
marioss andrew. cmu, edu
ABSTRACT
Vision-Langagg-Actiin (VLA) models show promise in embodied reasoning, yet remain far from true generalists- often require task-specific fine-tuning, incur high compute costs, and generalize poorly to unseen tasks. We propose VLta a unified, backhone-agnostic post-training framework for efficient and scalable alignment. Meta VLA introduces Context-Aware Meta Co-Training, which consolidates diverse target tasks into a single fine-tuning stage while lever- aging structurally diverse auxiliary tasks improve in-domain generalization. Unlike naive multi-task SFT, MetaVLA integrates a lightweight meta-learning mechanism deehhne from Attentive Neural Processes tion from diverse contexts with minimal architectural change or inference over- head. On the LIBERO benchmark, Meta with six auxiliary tasks outperforms OpenVL. by up 8.0% long-horizon tasks, reduces training steps from 240K to 75K, and cuts GPU time by 766 These results show that scalable, low- resource post-training is achievable- the way toward general-purpose em- bodied agents. Code will be available.
enable rapid adapta-
INTRODUCTION
Recent years have seen rapid progress in embodied Vasion-Languuge (VLA) models, which are typically pretrained from Vision Langung Models (VLMs) and adapted via supervised fine-
tuning (SFT) (Kim al. 2024 2025 Hung al., 2025) reinforcement learning (RL) (Zhang) jet al. 2025 Li et al. 2025) to enable transfer to new embodiment tasks. In one line of work, a pretrained VLA backbone adapted autoregressively and discretely decode action tokens, trained on annotated demonstrations consisting of video or image observations paired with natural language instructions (Kim et al., 2024 Brohan et al. 2022 2023 Neill et al., 2024). In contrast, another line of research represents output actions as continuous vectors, using techniques such as diffusion
policies flow matching (Black al. 2024 Intelligence al., 2025a NVIDIA et al. (025).
Despite advances in new task adaptation, current VLAs are not yet true generalists still far from fully -at-of-the-box usability and reliant on alignment through post-training (Zhou al.) 2025 Wang et al. [025] Huang et al. 2025b Din et al.) 2025 Guruprasad et al. [2055] Ma et al.) 2025). Compounding this, post-training remains practically constrained by benchmarks with low per-task data. Current practice (Kim et al.) 2024) fine-tunes each downstream task independently, increas- ing overall training cost, hindering knowledge transfer across related tasks, and ultimately limiting success rate. These task schedules are often brittle: many gradient steps required be- fore stably meaningful action sequences emerge, raising the risk of poor generalization and slowing adaptation to new task variants. For example, OpenVL.A requires 240K training steps to fine-tune

---

Published as a conference paper at ICLR 2026
Swir1
SFT-43370
Sinver dute
Singles
Lapa--epp
oo Hakkr
Co-Pratted
Geves Bate
Fleeer dur
Noce3
Spetaal
fieeer dute
co 215 Srate Buikin Stepe
Mudac
(a) Higher success rate with fewer training steps. MctaVL.A achieves 4.4% higher average success rate while requiring 68.75% fewer train- ing steps compared to the OpeaVLA baseline on LIBERO benchmarks.
(h) Stronger cross-task general- ization with one single model. Open requires training sep- arate models for each task suite, resulting in higher training costs and poor cross-task performance. In contrast, MetaVLA achieves strong generalization across four suites with a single unified model.
(c) Faster convergence to higher accuracy across all target tasks. Comparison of training accuracy between MetaVLA and a baseline multi-task SFT when auxiliary tasks are added. MctaVLA consistently converges to higher accuracy across all LIBERO suites, while the baseline under- performs throughout training.
Figure 1: Three Key Merits of MetaVLA Compared to Bascline Approaches.
across all four LIBERO suites (OpenVLA Team 2024). while OpenVLA-OFT (Kim et al. 2025) demands approximately 1,50K "500 steps, including both diffusion and non-diffusion parts. Long- horizon tasks such as LIBERO-Long further dominate the training schedule and often become the system bottleneck.
While recent work (Black et al., 2024 Intelligence et al., 2025a Qu et al. 2025) has focused on
expanding datasets and exploring backbone architecture or training protocol innovations during pre- training. we instead tackle it from an orthogonal perspective at the post- post-training stage. Our experi- ments begin with a vanilla multi-task co- co-training setting: applying a standard SFT single model across related in-domain tasks (i.e., the four LIBERO suites). Indeed, we observe a reduction in total GPU training hours and improved success rates, which naturally motivates us to raise a ques- tion: can we introduce even more auxiliary tasks in the co-training to further boost VL. models? Sadly, we find that naively adding auxiliary tasks with greater domain diversity slows convergence and degrades performance. We attribute this surprise to the optimization instability arising from heterogeneous distributions, where misalignments in both the feature space (e.g., camera views) and action space (e.g., degrees of freedom) hinder the benefits of co-training.
Building on these ideas, we propose MetaVLA, a unified framework that fills a critical gap in VLA post-training by intelligently introducing auxiliary tasks without incurring the inefficiencies of per-task SFT or the performance drop of naive multi-task SFT. It introduces Context-Aware Meta Co-Training, which whichjointly trains all target tasks with a unified model, improving adaptation by lever- aging cross-task data through context bank. The context bank is memory-augmented mechanism that contains auxiliary knowledge with domain diversities derived from Attentive Neural Processes (ANP) (Kim et al. 2019) based on Meta-learning. This lightweight module injects out-of out-of-domain information gain without disrupting target optimization, enabling scalable and robust adaptation. Meta is maintenance-friendl friendll,, mackbone-agnostic and easily extends beyond SFT to training paradigms like reinforcement learning. Figure highlights three key advantages of Meta over existing approaches.
Experiments show that MetaVLA with six auxiliary tasks outperforms the Open baseline by 4.4% and multi-task SET by 3.1% average, with gains up to 8.0% on LIBERO-Long, It unifies training into single model, reducing steps from 240K to 75K and GPU time by 76%-from 100 to 24 hours. Despite its flexibility, the compact in latency. The following sections present our framework, setup, and results, showing how MetaVLA boosts convergence, efficiency, and action reasoning. Our main contributions are follows:
module adds only 0.3 ms/token
2

---

Published as a conference paper at ICLR 2026
We investigate underexplored direction: improving post-training efficiency and gener- alization ability through incorporating diverse auxiliary tasks with negligible optimization overhead.
We propose MetaVLA, a suite of plug-in module and training recipes that enables fast and scalable adaptation with strong generalization. MetaVLA is engineering-friemt and agnostic to backbone architectures and underlying training pipelines.
We conduct comprehensive experiments to show that MetaVIA delivers superior perfor- mance with significant efficiency gains by reducing model count and GPU training hours, while preserving fast inference.
2 RELATED WORK
2.1
VISION-LANGUGEE-ACIIII MODELS
Recent advances in LssionLLanguage-AAtion (VLA) models have been driven by supervised fine- tuning (SFT) of pretrained -ision-Languaa Models (VLMs) to map visual context and language in- structions to action sequences- stage we refer to as "pretraining" for VLA. These models are then
adapted via SFT (Kim et 2024
2025 Hung al. 2025) or reinforcement learning (RL) (Zhang
et al. 2025
Li et al. 2025) to unseen embodied tasks.
One line of work adapts pretrained VLA backbones autoregressively decode discrete action kens (Kim et al.) 2024, Brohan et al., (022); 2023) O'Neiill et al., (024)) while another represents actions as continuous vectors using techniques like diffusion policies and flow matching (Black et al., 2024; Intelligence et al.] 2025a NVIDIA et al. (025). For backbone design, recent studies explore alternatives such as Qwen2.5 VL (Bai et al., 2025 Qu et al.) 2025 Hung et al.) 2025). In parallel, efforts like EO-1 (Qu et al. 2025) introduce interleaved Vision- Text-Action training for-
mats, while COT-VLA (Zhao et al.] 2025). OLA (Lin et al. 2025) and ThinkAct (Huang
et al. 2025a) incorporate reasoning data into training. Salciency-focusee works aim to improve
VLA
training through better tokenization or streamlined architectures (Pertsch et al. 2025 Reuss
al.
2025).
However, these approaches trade performance for costly pretraining interventions and meticulous data curation- an impractical strategy in cssutrceconstraiiee or democratized settings. Moreover, achieving meaningful gains often requires careful design (Driess et al.] (025), incurring high hu-
man overhead. In contrast, our method operates entirely the post-training stage, is orthogonal to existing techniques, and agnostic to both backbones and training pipelines- enabling seamless integration into various pretrained models and training recipes, including SFT and RL.
2.2 MULTI-TASK CO-TRAINING
Co-training across tasks has long been used to improve generalization (Doersch & Zisserman 2017 Zhang & Yang 2021), scalability (Devlin et al., 2019: Sun et al.) [0200] McLean et al. 2025 and data efficiency (Aghajanyan al., 2021 Crawshaw (0200). More recently, has shown strong suc- cess in LLMS and VLMs. GPT-2 (Radford et al.) 2019). for example, leverages diverse pretraining sources (e.g., web pages, Wikipedia, news) for broad generalization. LLaVA (Liu et al. (023b), pioneering open-source VLM, uses multitask fine-tuning for multimodal alignment across conver- sation, captioning, and reasoning tasks. This trend continues in models like Qwen-3 (Yang et al. 2025), which expands co- co-training diversity by incorporating code, textbooks, and multilingual data across both pretraining and post-training Similarly, Molmo and Pixmo (Deitke et al. (0024) pro- vide detailed ablations co-training with varied data sources, demonstrating the benefits of task and domain diversity. These advances highlight co-training as a key driver of performance both pretraining and post-training stages.
Despite its effectiveness, co- co-training remains less explored in VLA, especially at post-training stage. While recent works (Kim et al. 2024 Team et al. 2024 Kim et al. 2025 Hung et al. 2025 Reuss et al. 2025) co-train during VLA pretraining, they afterwards still rely on task-specific fine-tuning for downstream adaption, missing the benefits of shared task structure for better generalization. This results in duplicated model checkpoints, costly maintenance, high total training steps and thus longer total GPU training hours. A few efforts have taken multi-task co-training for post adaption, but they
3

---

Published as a conference paper at ICLR 2026
Instruction:
Concatenate
"Open Bottom drawer the cabinet put bowl it"
Context Data
Gaussen Sample
Xo
Nen
Ael-AAttentonn
Ailf-A-tnniion
Action Decoder
Pretrained VLM Llama2-7B
Tri
Fu
Sin
Value
ViT
Cross
Attention
Action Head
Queryy
Action ANP
Figure 2: MetaVLA Architecture. VLA backbone married with Context-Aware Meta Co-Training Framework, where the context memory bank is composed of both in-domain target tasks and out- of-domain auxiliary tasks. We further detail the definitions of variables in Section 3.2.1
are not free lunch. (Black et al. 2024) and Wo.5 (Intelligence et al. 2025a), require prohibitively expensive pretraining, while EO-I Qu et al. 2025) incurs high inference latency.
In contrast, our method systematically explores efficient task-shared adaptation in the post-training stage. introduces plug-and-play meta-learning module that enables scalable integration of un- seen auxiliary tasks, enriching target learning with diverse signals. The approach is backbone- agnostic and streamlines efficient generalization across tasks, achieving strong performance gains via a lightweight, maintenance- friendly co-training paradigm.
2.3
Meta-learning enables models to quickly adapt to new tasks, often using diverse contextual data and through episodic training (Finn et al., 2017, Koch et al., 2015 Santoro et al.) 2016 Ravi & Larochelle 2016). Attentive Neural Processes (ANP) (Kim et al. 2019), an amortized meta-learners inspired by Gaussian Processes, learn distribution over functions conditioned and target-specific latent vectors via attention mechanism (Vaswani et al. 2023). ANP is well-suited for VLA due to its task-invariance, selective attention to relevant demonstrations, and avoidance of direct context optimization during adaptation. These properties simplify cross-domain training, enhance stability, and enable scalability- for leveraging auxiliary data effectively, as shown in later results.
both global prior
3 METHOD
3.1 TASK DEFINITION AND BACKBONE SELECTION
Our goal is to develop an efficient one-for-all VLA post-training paradigm capable adapting to diverse novel tasks unseen during pretraining.
Specifically, we adopt the LIBERO (Liu et al. 2023a benchmark as our set of target tasks and use Open (Kim al. 2024) the backbone. Nevertheless, our method is backbone-agnostic and can be seamlessly integrated with other pretrained VLA models. See Section 4.1 for further details.
3.2 METAVLA
Met employs Context-Aware Meta Co-Training approach that jointly trains on all in-domain suites with a single model, while leveraging contextual demonstrations through meta-learning. We formalize this mechanism by showing how Action-ANP offers principled way to aggregate and condition on heterogeneous context data, using it to enhance stability during co- co-training across task suites.
4

---

Published as a conference paper at ICLR 2026
3.2.1 ARCHITECTURE
To improve convergence and generalization in low-data task adaptation, we base our architecture on Attentive Neural Processes (ANP) (Kim et al.) 2019) meta-learner inspired by Gaussian Pro- cesses that models a distribution over functions conditioned on both context and target representa- tions. These latent codes capture global and task-specific semanties, aggregated via self-attention and cross- -ooss-attention, respectively.
By attending both current target data and related contextual data with ANP, the context data aggregated and encoded into two separate branches: (1) a deterministic representation that captures target task dependent information by attending target tasks to context tasks; (2) a stochastic global representation that models information independent of target task based on context distribution. These two representations offer the model a referable demonstration for the current prediction.
We introduce a compact module, Action-ANP. integrated into the Ilama2 (Touvron et al. 2023) action decoder. Following the original ANP formulation, Action-ANP first applies self- attention /Vaswani et al. (2023) across context examples to extract a global prior, which is then fused with target queries through cross-attention (Vaswani al. 2023) to form task-awart hybrid represen- tations. Formally, given the target feature Zr, contextual feature-action pairs (ICr, UCi) E (æc. vc), Action-ANP models the conditional distribution of functions over target action y/ given global and task-specific observations:
Miyr|xr,xc.Yc) plyrlxr, ((2(3c) (((Bc))))))))))))))))))))))))
Here, roe re and SC: E So per-context representations aggregated from all contexts data pairs (ICs UC) through self-attention. ir is the cross-attention output of query øt with context keys aC; and values rci. $c the mean of all Sci, while is stochastic latent drawn from the approx- imate posterior ((((5c) computed over the context. During training, an additional condensed target representation 5r is produced by the same self-attention and mean process as for šc, with ground truth pair (Tr, Vr). By reparameterizing the Gaussian latent the training objective maximizes a variational lower bound:
log fogp(yy/r,, Xc, 2
Botalar)/ooga Dog Pr, 2)] DKL (g((3r)) (=
This formulation enables MetaVLA to reconstruct target actions, regularized by a KL divergence that prevents the target distribution from drifting too far from the context distribution.
Unlike standard ANP, which uses smaller-scale neural networks, we integrate a pretrained Llama- 2 (Touvron et al. 2023) backbone from Open VLA. Action-ANP generates both stochastic and de- terministic contextual latent vectors, which are concatenated with the Llama hidden states before the final output layer. The combined representations are then passed through the LM head to pro- duce output logits, enabling end-to-end training via standard Llama decoding. See Figure 2 for an overview of the framework. We further summarize the symbols and definitions in Table
3.2.2 DATA BANKS
In our setup, there are two data banks: context bank and target bank.
For context bank, which acts as an external memory, it's composed of both in-domain tasks, which are four LIBERO suites in our case, and auxiliary tasks. For in-domain tasks, the four LIBERO suites (Liu et al. 2023a) are split into non-overlapped context sets and target sets. For auxiliary tasks, choose the large collection of partially open-sourced GROOT data (NVIDIA et al. 2025). A unified context bank then aggregates context sets from in-domain datasets and selected tasks from the auxiliary data. Details about auxiliary task selection will be discussed in Section 3.3
The target data bank contains only the target sets of in-domain tasks in our case, the task sets across all four LIBERO suites. Unlike standard VLA SFT, which trains separate model for each suite, our meta co-training strategy trains a single model across all target suites, improving scalability, generalization, and efficiency.
5

---

Published as a conference paper at ICLR 2026
3.2.3 TRAINING PROTOCOLS
To ensure broad contextual coverage, we refresh the context set every training steps. Specifically, at each multiple of K, we randomly sample be examples from each context task's dataset, keeping be consistent across tasks for simplicity. We set K = 200 to balance training speed and decoding quality, and choose by 32 optimize memory usage and performance. An ablation study on b is provided in Section
3.3 AUXILIARY TASKS SELECTION
To enhance context diversity and strengthen meta-learning, we in- troduce auxiliary task se-
Front View, 7 DoE, Single Arm
Side View 7 DoE, Single Arm
Front View, 1n DoE, Two Arms
lection mechanism. Specifi- cally, we incorporate the GROOT dataset (NVIDIA et al.] 22025] NVIDIA 2025) into the context bank for two key reasons. First, GROOT entirely unseen dur- ing OpenVLA pretraining, mak- ing it a valuable source of ad- ditional information gain. Sec- ond, it offers partial domain rel- evance to LIBERO while differ- ing structurally striking a bal- ance between familiarity and di- versity.
LIBERO
Auxiliary Training Data
Figure 3: Comparison between auxiliary tasks and LIBERO evaluation benchmark. LIBERO tasks use third-person front- view images and 7-DoF actions for a single-arm robot. In con- trast, our auxiliary data from GROOT introduces variation through side-view observations and two-arm robot with 14-DoF ac- tions. MetaVLA benefits from this data diversity, while Open- VLA struggles with the domain mismatch.
LIBERO tasks feature Franka Emika Panda arm with a gripper and primarily use front-facing camera views. In contrast, se- lected GROOT tasks include bimanual manipulation using front views and single-arm manipulation with side views only. These variations are intentionally chosen to test the robustness and general- ization ability of MetaVLA,
example of task difference among these tree types is showing in Figure 3 and more examples are in Section A.2
Unlike Zhao et al. (025), which carefully select tasks highly similar to LIBERO, our method is less strict in data varieties in the context bank, and more robust to the diversity of auxiliary tasks which we believe would introduce higher freedom for a more scalable adaption training framework. Experimental results show that MetaVLA, equipped with this multi-task co-training setup, achieves higher success rates and faster convergence across all LIBERO suites compared vanilla SaTebased co-training. Ablation study the effect of auxiliary task selection is presented in Section 4.4.3
EXPERIMENTS
4.1 EXPERIMENT SETTING
We evaluate our method against previous works on the LIBERO benchmark (Liu et al., 2023a). a Franka Emika Panda single arm simulation-based benchmark with four different task suites. The benchmark aims to evaluate the model's capability of generalizing to variations of the 500 expert demonstrations across 10 tasks provided for each task suite. LIBERO-Goal leaves objects and layouts unchanged, and varies by final task goals; LIBERO-Spatial keeps the objects and tasks un- changed, while re-arranging the layout; LIBERO-Objen uses the same layout environment, while changing the object types; LIBERO-Long (also known LIBERO-10) consists of long horizon tasks with a mixture of different distribution shifts above. Our method co-trains a single model for all four suites with up to 6 heterogeneous auxiliary tasks with panda gripper robots from GROOT dataset (NVIDIA 2025), a simulation dataset consisting of different robots and task types, More
details are discussed in Section 3.2.2 3.3 and A.2
We follow prior work (Kim et al. 2024 In-

---

Published as a conference paper at ICLR 2026
Model
Steps 30K
Goal
Spatial
Object
Long
Average
No.5 (Ineeliigence et al. 2025b
98.0
98.8
98.2
92.4
96.9
Diffusion Policy Chh al. 2023
68.3
78.3
92.5
50.5
72.4
ATM (Wen et al.
2023
77.8
68.5
68.0
39.3
63.4
LLA (Zheng al.
2025)
75.1
84.6
85.2
54.1
74.8
Open VLA Kim et al.
2024)
240K
76.2
84.7
87.0
51.8
74.9
SFT-ALIBERO
77.8
84.8
87.4
54.7
76.2
Iin=4 Iimannull
59.7
68.0
65.2
30.0
55.7
75K
SFT=44LBBER++33innge
24.6
16.8
9.7
1.5
13.2
3FT-4LLBERR--SSnneeellmmmm
15.2
5.6
12.0
1.6
8.6
SFT-LLIBERO+SSingeer-bnnaaau
187.5K
23.4
16.7
13.6
4.4
14.5
74.4
85.4
85.4
52.3
74.4
MctaVLA (ours)
78.9
88.5
88.5
55,3
77.8
SttohaA+SSochhatic (ours)
78.9
88.9
88.5
53.0
77.3
75K
(ours)
78.5
89.0
87.4
59.0
78.5
MetaVLA+3single (ours)
78.0
88.0
87.2
59.7
78.2
MetaVLAA+Ssinll+1IIinn Ibimanual (ours)
78.7
89.9
88.9
59.8
79.3
Table I: Success rate (%) and training step comparison with prior methods. All MetaVLA variants are single models trained for 75K steps. MetaVLA (ours) uses only LIBERO suites in the context bank without the stochastic module, while VeetaVLA+Stochenee
(ours) includes it. includes N single and M bimanual (two arms) auxiliary tasks described in Section 3.3 SFT-4LIBERO is a single-model baseline trained with vanilla multi-task SFT across all suites. OpenVLA (lop) comprises four Hugging Face models fine-tuned separately on LIBERO using the Open VLA-7B backbone, totaling roughly 240K steps. MetaVLA with six auxiliary tasks surpasses Open VLA by 4.4% and SFT-4LIBERO by 3.1% on average, with even larger gains on LIBERO-Long (8.0% and 5.1% respectively).
telligence et al. 2025b) and adopt Success Rate (SR) as our evaluation metric. Thanks to efficient co-training, our method requires only 24 hours to fine-tune across all four LIBERO suites using A100 80GB GPUs. We use VLA (Kim et al. 2024) as our backbone due to its completeness, maturity, and robust open-source codebase and evaluation pipeline, which has been widely adopted in the academic community.
To ensure fair comparison, we re-evaluate the Open baselines in the LIBERO simulation envi- ronment using the four single-task fine-tuned models from Hugging Face (Open VLA Team 2024), and adopt these as our baselines. Due to hardware variance and stochasticity, our results may slightly differ from the originally reported values (OpenVLA Contributors 2024). All the reported results on LIBERO are evaluated on one 24GB RTX-4090 GPU.
4.2 EFFECT OF VANILLA MULTI-TASK SET
As shown Table adding auxiliary tasks to vanilla multi-task SFT (SFT-4L tasks) consistently degrades performance. The degradation worsens as more tasks are added, sug- gesting the model struggles with domain shifts and fails to converge. Training convergence curves in further corroborate this finding. One possible factor is reduced training steps per task. For instance, in SecALLBBRR+Ssinggee werigll+-bimmmun trained for 75K steps, per-task steps drop from 18.75K (in SFT-4LIBERO) 7.5K. To test this, we increase training to 187.5K steps. While perfor- mance improves slightly, it remains well below VetaVLA-wit or without auxiliary tasks. Further- more, as shown Figure training curves at 187.5K steps across all three metrics metrics-Accuracyy -Accuracy Im- itation Loss, and L1 Loss signal its suboptimal adaptation. This supports view that Meta VLA scales more robustly, leveraging auxiliary data without encountering optimization instability. A more rigorous proof of this view is left future work due to computational constraints.

---

Published as a conference paper at ICLR 2026
4.3 EFFECT OF CONTEXT-AWARD META CO- TO-TRAINING
As shown in Table 1 MetaVLA- with without auxiliary tasks- outperformss all baselines, in- cluding OpenVLA baseline and SFT-4LIBERO, across all LIBERO tasks and on average. With six auxiliary tasks, it improves over Ope by 4.4% and SFT-4LIBERO by 3.1%, 1%, notably on LIBERO-Long, with gains of 8.0% and 5.1% %, respectively. Moreover, Meta VLA reduces model count one and cuts training steps from 240K 75K. Examples of success cases are demonstrated in Section A.6
4.4
ABLATION STUDY
4.4.1
EFFECT OF DIFFERENT BACKBONE
Model
Goal
Spatial
Object
Long
Average
NORA-Long (Hung al. 2025
85.4
90.5
95.0
70.6
85,4
DORA-Long-SFI--LIBS
87.0
92.5
94.0
75.5
87.3
DORA--Log-SSFT--LBBRRR055 hitnsniiilll---bmuaallll Ibimanual
73.6
79.5
75.2
37.2
66.4
MetaYLA-NORA-Long (ours)
90.8
96.2
96.5
77.8
90.3
MetaVLA-NORA- NeaaVLA-NORR-Loon=SSiinnn http==Siinneeebbbmmnee bimanual (ours)
93.8
95.8
97.2
80.2
91.8
Table 2: Success rate comparison applying Meta variants NORA-Long. NORA- Long-SFT-4LLBERRO is single-model baseline trained with vanilla multi-task SFT across all suites using NORA-Long. NORA-Long comprises four Hugging Face models fine-tuned separately on LIBERO using the NORA-Long backbone. Without auxiliary tasks, MetaVLA-NORA-Lon sur- passes NORA-Long by 4.9%, and exceeds NORA-Long-SFT-44 -FT--LIBR--O by 3.0%. Adding auxil- iary tasks further pushes the average success rate to 91.8%, achieving the highest performance in LIBERO-Goal, Object, and Long.
To validate MetaVLAA's effectiveness in various backbones, we assessed MetaVL.A variants on NORA (Hung et al. (025). 3B Dwen2.5-VLL-assed (Bai et al. 2025) VLA model. We selected NORA-Long (Deep Cognition and Language Research (DeCLaRe) Lab 2025) variant as the base model because it provides stronger LIBERO performance baseline than NORA. To ensure fair comparison, we re-evaluate the NORA Long baselines in the LIBERO simulation environment using the four single-task fine-tuned models from Hugging Face (Deep Cognition and Language Research (DeCLaRe) Lab 2025), and adopt these as our baselines.
As shown in Table without auxiliary data, MetaVLA outperforms NORA-Long by 4.9% erage and LORKA--Loo-SSF---IIBBEE erage success rate further improves by 6.4% compared to NORA-Long. For more challenging suites-Goal and Long-the improvements are 8.4% and 9.6%, respectively. Moreover, consistent with results using the VLA backbone, cantly outperforms its native SFT counterpart,
by 3.0%. When auxiliary tasks are incorporated, the av-
signifi- LoNNNooA--Loo---ninn-e------- 1imaleual,aa
by 25.4%. %. The training convergence curves on accuracy and loss in Figure 10 further bolster stronger stability of MetaVLAA during training as more diverse tasks are added. Together, these results demonstrate VL'ss backbone-agnostic capability.
4.4.2 EFFECT OF CONTEXT BATCH SIZE
As shown in Figure 4 success rate increases monotonically with batch size under our setting. A relatively small context batch size 32 yields the best performance, which doesn't 't introduce extra overhead to memory footprint. A detailed table is shown in Table 6lin Appendix.
4.4.3 EFFECT OF AUXILIARY TASK SELECTION
As shown in Table I MetaVLA outperforms SFT-4-IBERO counterparts across all three auxil- iary task settings, demonstrating robust generalization to variations in camera views, action spaces, and the number context tasks. These results highlight a promising opportunity to scale up the context bank.
8

---

Published as a conference paper at ICLR 2026
Succes Rate 1561
2.0
57
16.7
77.8
26
74.9
74.6
732
86
SI 1-LLIBLRD
11
MetaVLA
42
37
12
smrat BErR
Hor moooettttssa 4:
Menoitall -0)
MateicAb 11
Hetai 32)
Context batch size Goal
Context batch size Spatial
Context batch size Object
Context batch size Long
Figure 4: Left: Per-suite LIBERO success rate across varying context batch sizes. OpenVLA refers to the four Hugging Face baseline models, each fine-tuned individually on LIBERO using the Open VLA-7B backbone, while SFT-4LIBERO multi-task SFT across all suites. For each suite, success rate increases monotonically with context batch size. Right: Average success rate across LIBERO suites with varying confext batch sizes, OpenVLA denotes the four Hugging Face models baselines fine-tuned individually on LIBERO with the OLenVLA--B backbone, while SFT-4LIBERO multi-task SFT across all suites. be indicates the context batch size. Larger context batches consis- tently yield higher average success rates.
single-model baseline trained with vanilla
single-model baseline trained with vanilla
4.4.4 EFFECT OF PARAMETER SIZE CHANGE
To rule out the possibility that performance gains stem solely from increased parameter size, conduct an ablation in which the architecture remains unchanged, but the context bank is replaced to include only tasks-bridge orig and fracta/20220817 data-both part already included in the VLA pretraining dataset (OpenVLA Contributors) 2024). The result, denoted as VLA- Pretrained-Context-ONNYY in Table shows significant drop across all LIBERO suites compared to MetaVLA. This suggests that the performance boost is not simply due to increased parameter size, but rather stems from the full design portfolio along with the integration of exotic auxiliary tasks that enrich the context with diverse and informative signals.
4.4.5 EFFECT OF MULTI-TASK CO-TRAINING MECHANISM
To assess the impact of task-d co-training, we replace MetaVLA's full target set (all four LIBERO suites) with single suite at time. For simplicity, we adopt frugal context bank con- taining only the four LIBERO suites without auxiliary tasks-matching the setup in Table [ for MetaVLA. Under this setting, we train four models independently via SFT, one per suite, using the same total training steps (240K) as OpenVLA (OpenVLA Team 2024). We refer to this con- figuration as MetaVLA-EACH at 240K (final step) and 120K (mid-training) to highlight the earlier convergence benefits of MetaVLA.
For evaluation, we report results for both Open basclines and
Results in Table 3 reveal three key findings: (1) MetaVLA-EACH outperforms the Hugging Face VLe baselines (OpenVLA Team 2024) final steps; (2) it achieves higher success rates earlier in training across all suites; and (3) on complex suites (Goal, Long), performance continues improve, while simpler ones (Spatial, Object) converge earlier- sugges-nn that task diversity
benefits more challenging tasks.
These findings highlight the effectiveness of Action-ANP within scalable, memory-based meta- learning framework. However, compared to full MetaVLA (Table MetaVLA-EACH sacrifices unified generalization and training efficiency, requiring four models and more compute (120K vs. 75K steps).
4.4.6 EFFECT OF STOCHASTIC LEARNING
As shown the ELBO bound equation 2 Action-ANP jointly optimizes a reconstruction loss and a KL divergence term. In Table [] MetaVLA-Stochaeee includes this stochastic regularization, while MetaVLA does not. The stochastic variant improves performance on the Spatial suite, per- forms comparably Goal and Object, but underperforms on Long. Since the KL term encourages proximity between context and target distributions an assumption that may not hold in more com- plex settings-we hypothesize that the greater domain shift in Long tasks leads this performance

---

Published as a conference paper at ICLR 2026
Method
Total Steps
Steps
Goal
Steps
Spatial
Steps
Object
Steps
Long
OpenVT.A A- S.T.A-120K 120K (Kim al.
2024
120K
30K
71.4
10K
81.2
30K
85.8
50K
44.4
120K
30K
76.4
10K
86.1
30K
89.0
50K
55.4
OpenVLA-2240 24-0 (Kim et al.
2024
240K
60K
76.2
50K
84.7
50K
87.0
80K
51.8
MctaVLA-EAACH-ZUU
240K
60K
77.4
50K
85.8
50K
88.5
80K
55.8
Table 3: MetaVLA-EACHI Per-suite success rates across LIBERO. OpenVLA denotes the four baseline models fine-tuned separately for each LIBERO suite, released on Hugging Face and trained for 240K total steps. OpenVLA-120K follows the same setup but with 120K steps. MetaVLA- EACH-120K and HelaVLA-EACH-22 240K steps, respectively, without co-training. Thanks to the Action-ANP design, all Veta EACH variants outperform their Open counterparts with fewer steps. For Goal and Long, performance continues to improve 240K steps, indicating stronger learning potential.
are our models trained separately per suite for 120K and
drop. In contrast, the deterministic variant, which relies solely on reconstruction loss, provides more precise modeling, making it more effective for challenging tasks. For this reason, the stochastic module is disabled in all other MetaVLA experiments for practicality.
4.5 EFFICIENCY DISCUSSION
We evaluate all tasks using one RTX-4090 GPU with batch inference. Our method added slightly more trainable parameters to the original architecture due to its lightweight property, which only increases inference latency by 0.3 ms/token, shown in Figure 9 in Appendix. Moreover, it reduces total GPU training time by 76%-ffoo 100 to ~24 hours- by cutting training steps from 240K to 75K. It also consolidates four task-specific models into a single one, streamlining deployment and improving maintenance efficiency.
4.6 WHY DOES OUR METHOD WORK?
Multi-task co-training promotes knowledge sharing across related in-domain tasks, while Action- ANP leverages diverse auxiliary data boost target performance and mitigate optimization instabil- ity from domain shifts. As shown in Figure MetaVLA consistently outperforms naive multi-task Accuracy, Imitation Loss, and LI Loss. The first two assess the quality of generated discrete tokens, while LI Loss measures the resulting continuous actions for robot execution. These results demonstrate both the effectiveness and stability of our approach.
SFT across all three convergence
In Section 4.4.2, we observe a monotonic performance gain with larger context batch sizes, and in Section 4.4.3, steady improvement with more diverse auxiliary tasks. While we do not exhaust all combinations due to memory and compute constraints, these trends suggest the potential of Context Sealing-increasse batch size and task diversity in the context bank may further enhance target-task performance, Moreover, given Meta VLA's robustness to context diversity, augmenting the context bank with web-scale data previously explored only the pretraining stage (Black al., 2024
Intelligence et al., 2025a Qu et al. 2025 may offer additional benefits. We leave this to future work.
CONCLUSION
We presented MetaVLA, framework that addresses the inefficiencies and brittleness of current VLA post-training pipelines. By introducing Context-Aware Meta Co- Fo-Training, MetaVLA inte- grates auxiliary tasks without destabilizing optimization, achieving superior convergence speed, ef- ficiency, and generalization. MetaVLA is lightweight, blug-and-play, and backbone-agnostic mak- ing easy to extend beyond supervised fine-tuning to reinforcement learning or hybrid pipelines. Empirical results on LIBERO show consistent gains over both per-task fine-tuning and naive multi- task SFT, while significantly reducing training cost and model count. Looking forward, we envision extending Meta VLA to broader backbones and wider benchmarks, incorporating web-scale mul- timodal data, and deploying on real robots. We hope this work inspires future research toward efficient, scalable, and truly generalist embodied VIA systems.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finetuning, 2021. URL https:
www.iv.orgaass//..oooooo 11008
Shuai Bai, Keqin Chen. Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu. Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang. Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
URL /tttps:/aaaatt
org/ abs/2502 11222.199 13923
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. To: A vision- language-action flow model for general robot control, 2024. URL hhttps://aae
org/
abs/2410. 24144.216661
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J. Joshi, Ryan C. Julian, Dmitry Kalashnikov, Yuheng Kuang. Isabel Leal, Kuang-Huel Lee, Sergey Levine, Yao Lu, Utsav Malla, Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag R. Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Anand Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Vanhoucke, Steve Vega, Quan Ho Vuong, F. Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-1: Robotics transformer for real-world con- trol at scale. ArXiv, abs/2212.06817, 2022. URL htt.pttttss/pppp...ettt.ccloco.. www.aniiccnccoaaacaoa org/ To:
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Krzysztof Choromanski, Tianli Ding, Danny Driess, Kumar Avinava Dubey, Chelsea Finn, Peter R. Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexan- der Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil J. Joshi, Ryan C. Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Sergey Levine, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael S. Ryoo, Grecia Salazar, Pannag R. San- keti, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Van- houcke, Quan Ho Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Ted Xiao, Tianhe Yu, and Brianna Zitkovich. Rt-2: lisionllaggaage-action models transfer web knowledge to robotic control. ArXiv, abs/2307.15818, 2023.
URL https: /tas:://ppppi www.ntetccccchhoooaccc. org/
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, pp. 027836492412736668 2023.
Michael Crawshaw. Multi-task learning with deep neural networks: A survey, 2020. URL https: //arxiv..rr//bb///009..077
Deep Cognition and Language Research (DeCLaRe) Lab. Nora fine-tuned models on libero tasks.
2025. Checkpoints
used: Goal: httpl//ssshgguullc....aa/////al. Spatial: nora- f
LIBERO-10: nora- forz==Lonn-
https///aogiagieee.///eeeeeeee.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jac Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin
11

---

Published as a conference paper at ICLR 2026
Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison- Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng. Jon Bor- chardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi. Molmo and pixmo: Open weights and open data for state-of- the-art vision-language models, 2024. URL https: /tt::///rxxii v.x.iv.oo org/ abs/ 22//244990 17146
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv..
abs/ 1ees/1810.o 011810000 04805
Muhayy Ud Din, Waseem Akram, Lyes Saad Saoud, Jan Rosell, and Irfan Hussain. Vision language action models in robotic manipulation: A systematic review, 2025. URL htttps:/aaxix: 20000/2507.100 10672
Carl Doersch and Andrew Zisserman. Multi-task self-supervised visual learning, 2017. URL
bttps::/arxiis wttg://///////o//oo Bttp:s//exxX...rrr////1110.... 07860
Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, and Sergey Levine. Knowledge insulating vision-language-action models: Train fast, run fast, generalize better, 2025. URL https:// arxiv.org/abe.
23705
Chelsea Finn, P. Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks, In International Conference on Machine Learning, 2017. URL https:
//api.semanticscholaa http://ooo.aaooooooooooooaao/e..
Pranav Guruprasad, Yangyue Wang, Sudipta Chowdhury. Harshvardhan Sikka, and Paul Pu Liang. Benchmarking vision, language, & action models in procedurally generated, open ended action environments, 2025. https://p http///axvvv... org aat.ab///5555 05540
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https:
09685
Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Yu-Chiang Frank Wang, and Fu-En Yang- Thinkact: Vision-languaag-action reasoning via reinforced visual latent planning, 2025a. URL
httpps:/arrxv..orgg// wrtp:///rraabbb//////e/// 16815
Huang Huang, Fangchen Liu, Letian Fu, Tingfan Wu, Mustafa Mukadam, Jitendra Malik, Ken Gold- berg, and Pieter Abbeel. Otter: vision-languagg--attion model with text-aware visual feature extraction, 2025b.
03734
Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, and Soujanya Poria. Nora: A small open-sourced generalist vision language action model for embodied tasks, 2025. httpps:://aaiii.rrrr.rr
/504. 19854
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling. Haohuan Wang, Lili Yu, and Ury Zhilinsky. no.5: vision-language-action model with open-world generalization, 2025a. WtLhhttp:: org/abs/250s 16054
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z.
12

---

Published as a conference paper at ICLR 2026
Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haobuan Wang. Lili Yu, and Ury Zhilinsky. To.s a vision-language- action model with open-world generalization, 2025b. URL arg/abs/2504 6055
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, S. M. Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. Attentive neural processes. ArXiv, abs/ 1.0611, 2019. URL /tas://appi. semant wwweemnnnischhooooo www.ollaa..rrgoooouuooooee
58014184
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Ben- jamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source Asioo-language-actionn model. arXiv preprint 2tw.:iv:24000944666 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-languagg--ction models: Opti- mizing speed and success, 2025. URL
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2, pp. 1-30. Lille, 2015.
Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, Zhaohui Yang, Kaiyan Zhang, Xuckai Zhu, Yuchen Zhang, Tianxing Chen, Ganqu Cui, Dehui Wang, Dingxiang Luo, Yuchen Fan, Youbang Sun, Jia Zeng, Jiangmiao Pang, Shanghang Zhang, Yu Wang, Yao Mu, Bowen Zhou, and Ning Ding. Simplevla-rl: Scaling vla training via reinforcement learning, 2025. URL https://arxii
org htg/abs//20 2509. 09674
Fanqi Lin, Ruiqian Nai, Yingdong Hu, Jiacheng You, Jumming Zhao, and Yang Gao. Onetwovla: A unified vision- laaiion-anguage-cction model with adaptive reasoning, 2025. htttps:::////xii
org/abs/ 2505. 11917
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qian Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. ArXiv, abs/2306.03310, 2023a. http:/ooppssD::5588555
URL https:tp////mc //ppi. Semant www.nt
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023b. URL https://arxivvoor//aaaa 08485
Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey vision- wwtp://wwww..../////../.// language-action models for embodied ai, 2025. URL D 14093
Reginald McLean, Evangelos Chatzaroulas, Jordan Terry, Isaac Woungang, Nariman Farsad, and Pablo Samuel Castro. Multi-task reinforcement learning enables parameter scaling, 2025. URL
https://aaxxvv.rrr//////5......
NVIDIA. hht.epallllarooooooooooooeeebeoi dataset. bbbasstttttpes:/uugggggggeeee .c/ oateaerr/aadddd/hhsssaaaaatooo.a Retrieved August 31, 2025.
--w-----Bbbooeett-------- Sim 2025.
NVIDIA, :, Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li. Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loie Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang, Jiannan Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. Gr00t nl: An open foundation model for generalist humanoid robots, 2025. URL
fttps://arxxn org/ abs /ttp://iww.rggg//////3.1111111 14733
Open Contributors. Openvla: Open vision- larion--angugge-ace foundation model.
https://
g thub. wwtp://wwww.//pppeeeoaaa a 2024. Accessed: 2025-09-19.
13

---

Published as a conference paper at ICLR 2026
OpenVLA Team. Openvla-7b fine-tuned models on libero tasks. https//ttt//hggggggffccecc
openvia 2024. Checkpoints used:
Goal: /ctt://s/ooouuaaaae////r/o/////2
Object: http:///wwggggtt.c..ec/////////a LIBERO-10: http::/ttwtt.e.ooggaaoaoeaeee/e/ |tt.a///...///////aaaaa////////.
Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abra- ham Lee, Acor Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog. Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang. Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerick, Beomjoon Kim, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu, Charlotte Le. Chelsea Finn, Chen Wang, Chenfeng Xu, Cheng Chi, Chenguang Huang, Christine Chan, Christopher Agia, Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Driess, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman. Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Foster, Fangehen Liu, Federico Ceola, Fei Xia, Feiyu Zhao, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Ge Yan, Gilbert Feng. Giulio Schiavi, Glen Berseth, Gregory Kahn, Guanzhi Wang, Hao Su, Hao-Shu Fang, Haochen Shi, Henghui Bao, Heni Ben Amor, Henrik Christensen, Hiroki Furuta, Homer Walke, Hongjie Fang. Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jachyung Kim, Jaimyn Drake, Jan Peters, Jan Schneider, Jasmine Hsu, Jeannette Bohg, Jeffrey Bingham, Jeffrey Wu, Jensen Gao, Jiaheng Hu, Jiajun Wu, Jialin Wu, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jingyun Yang, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang. Jordi Sal- vador, Joseph Lim, Junhyek Han, Kaiyuan Wang. Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Ken Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Lin, Kevin Zhang. Kiana Ehsani, Kiran Lekkala. Kirsty El- lis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh, Kuo-Hao Zeng. Kyle Hatch, Kyle Hsu, Laurent Itti, Lawrence Yunliang Chen, Lerrel Pinto, Li Fei-Fei, Liam Tan, Linxi Jim Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Mem- mel, Masayoshi Tomizuka, Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang, Mingyu Ding, Minho Heo, Mohan Kumar Sri- rama, Mohit Sharma, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen, Nicolas Heess, Nikhil J Joshi, Niko Suenderbauf, Ning Liu, Norman Di Palo, Nur Muhammad Mahi Shafiullah, Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R Sanketi, Patrick Tree Miller, Patrick Yin, Paul Wohlhart, Peng Xu, Peter David Fagan, Peter Mitrano, Pierre Sermanet, Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Quan Vuong, Rafael Rafailov, Ran Tian, Ria Doshi, Roberto Martín- Martín, Rohan Baijal, Rosario Scalise. Rose Hendrix, Roy Lin, Runjia Qian, Ruohan Zhang, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante, Sean Kirmani, Sergey Levine, Shan Lin, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham Sonawani, Shu- ran Song. Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suneel Belkhale, Sungjae Park, Suraj Nair, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony Z. Zhao, Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vincent Vanhoucke, Wei Zhan, Wenxuan Zhou, Wolfram Burgard, Xi Chen, Xiaolong Wang, Xinghao Zhu, Xinyang Geng, Xiyuan Liu, Xu Liangwei, Xuanlin Li, Yao Lu, Yecheng Ja- son Ma, Yejin Kim, Yevgen Chebotar, Yifan Zhou, Yifeng Zhu, Yilin Wu, Ying Xu, Yixuan Wang, Yonatan Bisk, Yoonyoung Cho. Youngwoon Lee, Yuchen Cui, Yue Cao, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang, Yunfan Jiang, Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma, Zhuo Xu, Zichen Jeff Cui, Zichen Zhang, and Zipeng Lin. Open x- embodiment: Robotic learning datasets and rt-x models Open x x-embodiment collaborationO. In 2024 IEEE International Conference Robotics and Automation (ICRA), pp. 6892-6903, 2024. doi: 10. 10.1100//CRA55744.... 00000024416611 1477.
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision- Iasion-lagguage-action models, 2025. URL htt.httta://// pectta://aara org abs 22501 09747
14

---

Published as a conference paper at ICLR 2026
Delin Qu, Haoming Song, Qizhi Chen, Zhaoqing Chen, Xianqiang Gao, Xinyi Ye, Lv, Modi Shi, Guanghui Ren, Cheng Ruan, Maoqing Yao, Haoran Yang, Jiacheng Bao, Bin Zhao, and Dong Wang. Embodiecdonevision: Interleaved vision-text-ace pretraining for general robot control, 2025.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Sachin Ravi and H. Larochelle. Optimization as model for few-shot learning. In International Conference on Learning Representations, 2016. URL
ID: 677 67413369
Moritz Reuss, Hongyi Zhou, Marcel Rühle. Ömer Erdinç Yağmurlu, Fabian Otto, and Rudolf Li- outikov. Flower: Democratizing generalist robot policies with efficient vision-language-acioon /2509. 22099009999
flow policies, 2025. htttltttss//aii..oootrttt
Adam Santoro, Sergey Bartunov, Matthew M. Botvinick, Daan Wierstra, and Timothy P. Lill- icrap. Meta-leaming with memory-augment neural networks. In International Conference on Machine Learning, 2016. URI. htt.hhttss//aa www.ste cectoticcchllal.c ar. orw.aarrrrr//ooosttt......
6466088
Ximeng Sun, Rameswar Panda, Rogerio Feris, and Kate Saenko. Adashare: Learning what to share for efficient deep multi-task learning, 2020. URL
12423
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405. 12213, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. Uttptttss://///ttt http///rxvvvv/// org. abs 2307 09288
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL
orq/abs/1700 03762
Wang, M. Leonard, K. Daniilidis, D. Jayaraman, and E. S. Hu. Evaluating piO the wild: Strengths, problems, and the future of generalist robot policies, 2025. URL : / //penn-pal- Iab qithub. Lttp/pi-EExpe iiweerrieeetttiieee in- wwwnttheeeiete -Widd
Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning, arXiv preprint arXi::2401.00025, 2023.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng. Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghho Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL http::t/////........ / Parxii org/ abs 2505. 09388
15

---

Published as a conference paper at ICLR 2026
Yu Zhang and Qiang Yang. A survey multi-task learning, 2021. URL httttttt//xxxi..coo
abs/ abs/1707 0es1/777.08114
Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, Dieter Fox, and Huaxiu Yao. Grape: Generalizing robot policy via preference alignment, 2025. URLhttts::/aat arxiv org/ abs
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, Ankur Handa, Ming- Yu Liu, Donglai Xiang, Gordon Wet- zstein, and Tsung- Yi Lin. Cot-vla: Visual chain-of thouhht reasoning for vision-kanguage--cttoo models, 2025. URL httph:t/p///tttt wwwpaiwwwww..aaaaaeeeeeeooeeeeee hrrg/abs/ae abaaabss/2250 2
Ruijie Zheng, Yongyuan Liang. Shuaiyi Huang, Jianfeng Gao, Hal Daumé III. Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies, 2025. URL
10345]
Jiaming Zhou, Ke Ye, Jiayi Liu, Teli Ma. Zifan Wang, Ronghe Qiu, Kun-Yu Yu Lin, Zhilin Zhao, and Junwei Liang. Exploring the limits of vision-language-action manipulations in cross-task generalization, 2025. URL attps://arzxs
org/ 2ttp://aaxx...or/////250........ 15660
A APPENDIX
A.1 TRAINING CONVERGENCE
Figure presents Training Accuracy, Imitation Loss (cross-entropy generated discrete action tokens), and [.1 Loss (on the transformed continuous actions) for three auxiliary task settings: Isin- Ilmannal, 5single+ 1iimanall, and 3single. In all cases, MetaVLA consistently converges to higher performance across all three metrics.
A.2 CONTEXT TASK DETAILS
We use the LIBERO
dataset (Liu et al. 2023a) as both target and context tasks, and auxiliary context tasks only. detailed breakdown of the datasets is provided in Table Example tasks from LIBERO and GROOT are visualized in Figures 7 and
GROOT (NVIDIA 2025
respectively.
Dataset
Tasks LIBERO- Goal
LIBERO -Spttil
LIBERO Liu et 2023a)
LIBERO-Obie
LIBERO-L -Lng
bimanual .andaa www.alal-pnaaaaapapppeeee Threading
single_pande wappaaagnippeeeeeeee www.ggpee.popeeeeeeeeee
CoffeeServeMug
single panda gripper. Openneree
GROOT (NVIDIA 2025
single panda gripper. PricbeerPPPCCa
hwt.ppanaaggppppppee papaaddagippeePP/CCuurror/MMiii PnPCoouneeeT/o/iiirrwwa langleepanda_gnee htw.indagriippe..uSSnSSSSSp....
Table 4: Summary of datasets and tasks used in the experiments.
A.3 MODEL ARCHITECTURE AND TRAINING DETAILS
Model Architecture We build Open (im et al. 2024 the base model, integrating Action-ANP, a lightweight, memory-based meta-learning module. In Action-ANP. global prior rep- resentations encoded via self-attention, while tross-attention /Vawwani al. 2023) fuses target and context to produce a final hybrid latent representation. Each attention block is followed by Layer Normalization and a final MLP projection.
16

---

Published as a conference paper at ICLR 2026
SFT-4IEER0 "aigle of
SFT-AURERO- Tanga meraaaloo ine
Mus/LL Sinlle (anneelloerr
MOSOVLA- Taagge sgon To Tonnssoloous
OP 1-
SFT AL CLLIEEROO So sg o- moraaaaloon
Sengaa Somoraaniuuu
SIT-4L WOw.RRD--aaaaooeeeeocc
Merrst A-da POTANO)
Figure 5: Training convergence comparison for models trained with 75K steps. Training Ac- curacy. Imitation Loss, and LI Loss are compared between MetaVLA variants and SFT-4LIBERO under different auxiliary-task settings. All MetaVLA variants consistently converges to superior per- formance across all three metrics, while SFT-4LIBERO fails to adapt effectively the robustness and scalability of our approach.
17

---

Published as a conference paper at ICLR 2026
3.30
Acuracy
0.29
Taifing LO3S
0.5
3.0
2.0
230
3903 9X03 TH00 TEMEO 124EE3 19R003 172300 Palling
230 v05 79883 10000: Torning
17V809 193000 17N00
2000
1908 1eo 17405 1NDES Traning Iaration
Figure 6: Training convergence of MetaVLA with six auxiliary tasks (one bimanual and five single-arm) trained with 187.5K steps. All three metries metries-Accuracy, Imitation Loss, and LI Loss-converge to suboptimal levels.
Pick up the alphabet soup and place it the basket
Turn on the stove
LIBERO-Goal
LIBERO-Object
Pick up the black bowl on the wooden cabinet and place on the plate
Put the yellow and white mug the microwave close
LBERO-Spptia
LIBERO-Long
Figure 7: LIBERO examples. Each suite example includes frame from the primary camera view together with its task instruction.
Training Settings We trained all MetaVLA variants with LoRA (Hu et al. 2021 8 A100-80 GB GPUs with 75K training steps, taking approximately 24 GPU hours, using 8 80GB VRAM. Training hyperparameters in Table 5
A.4 EXPERIMENT DETAILS
A.4.1 INFERENCE EFFICIENCY
Our method is engineering orrienlring-friendly and computationally lightweight. We measure both token throughput and latency of the model end-to-end, VLA VLp Contributors 2024). All environments and packages are kept the same through- the experiment ensure fair comparison. Our efficiency results shown in Figure 9 Our
24GB RTX-4090 GPU against Open-
18

---

Published as a conference paper at ICLR 2026
Pick the mug from under the coffee machine dispenser and place it on the counter
Pick the thread and insert it into the ring
bimanual panda www.pgrrpppp.tiem Threading
_aree_aganaadaaa_____e_e_ CoffeeServeMug
Pick the lime from the cabinet and place the counter
Open the right drawer
Lingle_pandee panda_gripper..om OpenDrawer
pwtpppanniipparr....eee wwwpggee..PCaaaTooe.. PnoCutooCouuteee
Pick the com from the counter and place it in the microwave
Tum the sink spout to the right
single_pandee Saand_gripppr..Porn ottppper.PPPcctttteeooccooww
single_pandee _all__aaaaaadrdrrr_eaee -wwppprrgppgp... wtt.grgpper.rniiiissSe. TumSinkSpout
Figure 8: GROOT examples. Each task example includes frame from the primary camera view paired with its task instruction.
19

---

Published as a conference paper at ICLR 2026
Hyperparameter
Value
Shuffle Buffer Size
100000
FlashAttention-2
Enabled
LoRA Rank
32
LoRA Dropout
0.0
Total Batch Size
128
Gradient Accumulation Steps
Learning Rate
5c-4
Context Batch Size
32
Action-ANP Latent Dimension
2048
Table 5: Training Hyperparameters. Total batch size is computed 16 samples per GPU across
8 GPUs. Context batch size refers to the batch size used for each individual context task.
Action-ANP module introduces approximately 5.5% more latency compared to Open VLA, making MetaVLA an ideal practical choice achieving a higher success rate.
Throughput Comparison
Latency Comparison
200
181.8
Throughput (tokens/sec)
172.4
5,8
5.5
150
Latency (ms)
125
4
100
25
0
OpenVLA
MetaVLA
OpenVLA
MetaVLA
Figure 9: Efficiency Metrics. Our lightweight module only adds negligible overhead to inference cost, making MetaVLA practical for deployment and usage.
A.4.2 EFFECT OF CONTEXT BATCH SIZE
Table 6 shows the success rates of Meta VLA across different LIBERO tasks using different context batch sizes. The performance scales up we introduce more contextual data.
Method
Goal
Spatial
Object
Long
Average
VLE (Open VLA Contributors, 2024
76.2
84.7
87.0
51.8
74.9
SFT-4LIBERO
77.8
84.8
87.4
54.7
76.2
MetaVLA VLA(bc = =))
75.0
82.2
85.0
50.4
73.2
MetaVLA VLA(bc =8)
75.4
85.5
86.8
51.4
74.8
VLA(BC 166
76.8
87.8
88.0
54.3
76.7
Meta VLAbCC = =32)
78.9
88.5
88.5
55.3
77.8
Table 6: Effeet of different context batch sizes across different LIBERO task suites.
20

---

Published as a conference paper at ICLR 2026
A.4.3 EFFECT OF DIFFERENT BACKBONE
Figure [0 shows Training Accuracy and Imitation Loss for single and 1 bimanual auxiliary tasks when using NORA-Long (Hung et al. 2025) backbone. MetaVLA consistently converges to higher performance across both metrics compared to vanilla SFT, proving that our observation on Open- VLA (Open VLA Contributors 2024) is also true under another backbone.
1.0
Baseline-NORA-Long
MetavLA-NORA-Long
Tranning Accurazy
2.5
Training
64
1.8
Baseline-NORA-Long MetaVLA-NORA-Long
100 1833 13:33
2930
2.00
Training
Training
Figure 10: Training convergence comparison for NORA-Long backbone with and without MetaVLA after adding 5 single and 1 bimanual auxiliary tasks. Training Accuracy and Im- itation Loss are compared between MetaVLA variants and baseline SFT-4LIBERO under Ssin- gle+I co-training settings, Without MetaVLA, vanilla SFT with auxiliary tasks fails to adapt effectively, while proposed MetaVLA consistently achieves better accuracy and lower loss throughout the training.
A.5 SYMBOLS AND DEFINITIONS
We summarize all the symbols used in our Meta VLA architecture in Table 7
A.6 SUCCESS CASES IN LIBERO SIMULATION
Figures
13 and 14 demonstrate example execution sequences of MetaVLA successfully completing one task from each LIBERO suite in its simulation: Goal, Spatial, Object, and Long.
A.7
LLM USAGE
We used LLM to aid and polish writing.
21

---

Published as a conference paper at ICLR 2026
Figure 11: MetaVLA Execution Sequence Example on LIBERO-Goal, Instruction: Open the middle drawer of the cabinet
22

---

Published as a conference paper at ICLR 2026
Co
Coo
Coo
Cog
Coo
C
300
!oo
Oe
Figure 12: MetaVLA Execution Sequence Example on LIBERO-Spatial. Instruction: Pick up
the black bowl between the plate and the ramekin and place the plate
23

---

Published as a conference paper at ICLR 2026
Figure 13: MetaVLA Execution Sequence Example on LIBERO-Object. Instruction: Pick up the cream cheese and place it in the basket
24

---

Published as a conference paper at ICLR 2026
Figure 14: MetaVLA Execution Sequence Example on LIBERO-Long. Instruction: Put the
black bowl in the bottom drawer of the cabinet and close
25

---

Published as a conference paper at ICLR 2026
Symbol
Name
Definition
XT
Target feature
Encoded input feature for target queries.
Yr
Target action
Action corresponding to tar- get query X7.
XC
Context features
Encoded input features for context queries.
Context actions
Action corresponding to context query xc- pair
(rcc, yC:)
Context pair
A single from the context bank. Self-attention rep. for con- text pair i.
rCi
Deterministic context rep.
TT
Deterministic target rep.
Cross-attention rep. of attending to (#cs)-
sC
Stochastic context rep.
Self-attention rep. used to compute latent posterior.
Sc
Mean stochastic context rep.
Averaged stochastic repre- sentation over all context rep. {sci).
Sr
Mean stochastic target rep.
Averaged stochastic rep. over all target rep. {srv}.
Latent variable
Stochastic latent sampled from approximate posterior. Approximate posterior over 2 conditioned on context. Training posterior used for variational objective.
<x]B3c)
Context posterior
q(2/5T)
Target posterior
plyr|xr,rr, FT,
Decoder likelihood
Conditional distribution pre- dicting target actions. Regularizer aligning target and context posteriors.
DRI.--I-)
KL divergence
Table 7: Symbol table for Action-ANP and MetaVLA. rep. is abbreviation for representation.
26


Reviews:
[{"review_id": "bpCkCTruR7", "reviewer": "Reviewer_dnnY", "summary": "This paper introduces MetaVLA, a unified framework designed to improve post-training efficiency and generalization in VLAmodels. Unlike prior work that focuses on larger datasets or architectural changes, MetaVLA targets the post-training stage, addressing the limitations of naive multi-task SFT, which struggles with convergence and degraded performance when tasks differ widely in domain or action space.\n\nMetaVLA introduces Context-Aware Meta Co-Training, which intelligently integrates auxiliary tasks through a memory-augmented mechanism inspired by Attentive Neural Processes. This approach allows the model to leverage out-of-domain information without destabilizing optimization, effectively combining the benefits of multi-task learning and meta-learning. The framework is backbone-agnostic, easy to integrate, and applicable beyond SFT to reinforcement learning setups.\n\nExperiments on the LIBERO benchmark demonstrate MetaVLA’s strong performance and efficiency, compared to OpenVLA and multi-task SFT baselines.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes some empirical originality by integrating Attentive Neural Processes (ANP) into a large-scale VLA architecture, introducing Action-ANP, a compact module that enhances meta-learning for low-data task adaptation. This integration allows the model to capture both global and task-specific semantics through self- and cross-attention, improving convergence and generalization.\n2. Context-Aware Meta Co-Training effectively combines in-domain and diverse auxiliary tasks, leveraging an external context bank that broadens the learning signal without destabilizing optimization. The framework is able to train a single scalable model instead of separate ones for each task, and is robust to task diversity, handling heterogeneous data.\n3. Experimental results show that MetaVLA scales more robustly, leveraging auxiliary data without encountering optimization instability.\n4. Authors perform comprehensive ablations to evaluate effectiveness of design choices in each part.", "weaknesses": "1. Since the auxiliary task selection part incorporates additional data, such as GR00T dataset and tasks, I wonder whether performance gain of this co-training paradigm comes in architecture design, i.e. attentive neural process, or better usage of more training data?\n2. How generalizable is the proposed ANP architecture?Will this be only applicable to Llama 2 model or any other open source LLMs?\n3. I'm a little confused about the efficiency argument in the paper. Since during post-training, more training samples are included and seen by VLA model through auxiliary tasks, it seems that this will increase training cost. Are there statistics about how much time does training/inference time take?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bRjJc098zi", "reviewer": "Reviewer_rKvL", "summary": "This paper adopts the ANP approach from the field of meta-learning, effectively addressing the issue in VLA models that requires separate training for tasks with large distribution differences, thereby improving training efficiency and reducing the training costs of multiple models. Furthermore, the MetaVLA approach even enables mutual enhancement across different tasks, which is promising for the general multi-task embodied agents.", "questions": "1.\tIn Table 1, performance with ‘single 3’ is worse than with 1. How does the method scale? When more auxiliary data from diverse sources are used, does performance improve consistently? If so, where might the performance plateau?\n2.\tIt would be valuable for the authors to conduct experiments in embodied settings beyond robotic arm manipulation.\n3.\tWhy does SFT-4LIBERO outperform OpenVLA in Table 1? It is generally believed that individual suite-specific training is necessary for optimal performance, yet even a naive co-training approach via SFT surpasses individually trained models. What explains this result?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper innovatively applies methods from the meta-learning domain to address the cross-task generalization problem in Vision-Language-Action (VLA) models. While VLA research has long focused primarily on compositional generalization, this work provides a valuable direction for future research on cross-task generalization.\n2.\tThe use of meta-learning significantly reduces computational overhead, enabling a single model trained across multiple task suites to achieve performance comparable to models individually trained for each suite. This component can thus eliminate a large amount of redundant and labour-intensive training.\n3.\tThe core ANP component introduced in the paper is decoupled from the backbone architecture, making it highly modular and extensible, and easily adaptable to various VLA model frameworks.", "weaknesses": "1.\tExtensive experiments are conducted only on the LIBERO+GT000 setting. However, the method should be evaluated on more diverse cross-task datasets and auxiliary data sources to demonstrate its generalizability better.\n2.\tThe explanation of the ANP method in the paper is difficult to follow. It would be better if there were a more detailed exposition. Additionally, the concept of \"Context-Aware Meta Co-Training\" is not introduced until the Method section; it should be presented earlier in the paper to aid reader comprehension.\n3.\tThe ablation studies do not address key concerns. For example, under what conditions do subtask co-training mutually enhance each other, and when might they degrade performance? Furthermore, how does the domain similarity of different auxiliary data sources affect task synergy?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U1j6v33u7q", "reviewer": "Reviewer_hdkd", "summary": "This paper proposes MetaVLA, a post-training framework for fine-tuning vision-language-action (VLA) models. The approach builds on the Attentive Neural Process (ANP) paradigm, maintaining a context bank of demonstrations drawn from both in-domain and auxiliary tasks. During training, MetaVLA leverages the ELBO objective of ANP to reconstruct action sequences while regularizing them toward the underlying demonstration distributions. The method is evaluated on the LIBERO benchmark across four task suites, where it outperforms the OpenVLA baseline with fewer training steps.", "questions": "Please see my comments above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The paper introduces a formulation for post-training VLA models through a single unified model, reducing the need for per-task fine-tuning.\n2. The paper reports both training and inference costs, providing a clearer view of the method’s computational efficiency.", "weaknesses": "1. The motivation for few-shot meta-learning is to handle data-scarce scenarios, yet this paper builds its context bank from the entire dataset. Although it is sampled at the size of 32, it means full data access is still required. If the full dataset is available, direct supervised fine-tuning is equally feasible. In addition, the main results show that standard SFT performs only about 3% worse than the proposed method, even without the additional GR00T dataset used by the authors. An experiment of using truly few-shot data could strengthen the paper's arguments.\n2. The comparison excludes stronger, more recent fine-tuning baselines such as OpenVLA-OFT [1], which reaches ~97% average success on LIBERO with significantly faster inference, whereas MetaVLA achieves only 79%. A comparison between these 2 methods could clarify the paper's contribution.\n3. The claimed generalization is tested only within the four LIBERO suites, which are all visually and semantically similar. Evaluation on cross-benchmark transfer tasks, or more diverse embodied tasks (e.g., RLBench, ManiSkill, FrankaKitchen) would be helpful.\n4. The method is described as “backbone-agnostic,” yet experiments are conducted solely on the OpenVLA backbone. No evidence is provided that the approach transfers to other architectures.\n5. The presentations need substantial improvements. For example, several citation formats are incorrect; when the author or publication name is not mentioned in the text, citations should appear in parentheses. Also, the definitions of variables such as x, y, s_C are unclear. Figure 2 implies inputs are image observations and text instructions, yet these symbols seem to be something else. The experimental notation (e.g., “SFT-4LIBERO + 5single + 1bimanual”) is not defined in the description/ text, making it difficult to interpret results.\n\n[1] Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success, RSS 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iwj8DEnAty", "reviewer": "Reviewer_3qCv", "summary": "The paper presents MetaVLA, a unified post-training framework for Vision–Language–Action (VLA) models that introduces Context-Aware Meta Co-Training (CAMC). By integrating a lightweight Action-Attentive Neural Process (Action-ANP) module, the method allows a single model to leverage both in-domain and auxiliary task contexts to achieve efficient multi-task adaptation. Experiments on the LIBERO benchmark show consistent improvements over OpenVLA and vanilla multi-task SFT.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper addresses a highly practical and underexplored problem—efficient post-training of VLA models—with a clear motivation and well-articulated methodology.\n2. The proposed Action-ANP module is well integrated into the existing architecture and effectively stabilizes multi-task optimization without increasing inference complexity.\n3. The experiments are thorough and convincing, showing clear improvements in both success rate and efficiency with detailed ablations that support the claims.", "weaknesses": "1. The method is evaluated only on the OpenVLA-7B backbone, leaving its claimed backbone-agnostic property unverified.\n2. The inclusion of large-scale π₀․₅ results in Table 1 could confuse readers, as those models operate under significantly different training regimes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)