Interleave-VLA: Enhancing Robot Manipulation with Image-Text Interleaved Instructions
Authors: Cunxin Fan
Xiaosong Jia
Yihang Sun
Yixiao Wang
Jianglan Wei
Ziyang Gong
Xiangyu Zhao
Masayoshi Tomizuka
Xue Yang
Junchi Yan
Mingyu Ding
Venue: ICLR 2026 Poster (2026)
Keywords: Robot Learning
Foundation Model
Multimodal Learning
Primary area: applications to robotics, autonomy, planning
TL;DR: We introduce a vision-language-action paradigm that understands interleaved image-text instructions and generates continuous actions, improving zero-shot generalization in real-world robotics with a large-scale interleaved embodied dataset.

Abstract: The rise of foundation models paves the way for generalist robot policies in the physical world. Existing methods relying on text-only instructions often struggle to generalize to unseen scenarios. We argue that interleaved image-text inputs offer richer and less biased context and enable robots to better handle unseen tasks with more versatile human-robot interaction.
Building on this insight, we introduce Interleave-VLA, a robot learning paradigm extending interleaved image-text instructions from digital world to directly generating continuous action sequences in the physical world.
Interleave-VLA offers a natural, flexible, and model-agnostic paradigm that extends state-of-the-art vision-language-action (VLA) models with minimal modifications while achieving strong zero-shot generalization.
Interleave-VLA also includes an automatic pipeline that converts text instructions from Open X-Embodiment into interleaved image-text instructions, resulting in a large-scale real-world interleaved embodied dataset with 210k episodes.
Comprehensive evaluation in simulation and the real world shows that Interleave-VLA offers two major benefits: (1) improves out-of-domain generalization to unseen objects by 2× compared to text input baselines, (2) supports flexible task interfaces and diverse instructions in a zero-shot manner, such as hand-drawn sketches.
We attribute Interleave-VLA's strong zero-shot capability to the use of instruction images, which effectively mitigate hallucinations, and the inclusion of heterogeneous multimodal datasets, enriched with Internet-sourced images, offering potential for scalability. [Our project site](https://interleave-vla.github.io/Interleave-VLA-Anonymous/) has more information.

Published as a conference paper at ICLR 2026
INTERLEAVE- VLA: ENHANCING ROBOT MANIPULA- TION WITH INTERLEAVED MMAGE-TEXT INSTRUC- TIONS
Cunxin Fan
Xiaosong Jia
Yihang Sun+
Yixiao Wang
Jianglan Wei
Ziyang Gong
Xiangyu Zhao
Masayoshi Tomizuka 3
Xue Yang!
Junchi Yan if
Mingyu Ding"
SCS & SAIS & SAT & ICISEE, Shanghai Jiao Tong University
2 Institute of Trustworthy Embodied AI, Fudan University
UC Berkeley LUNC-Chape Hill
Eqqual Contributions
Corresponneece Authors
Interleaved Embodiment Dataset
(b) Average Success Rate Interleave VLA and Text- VLA
Carmiiing
Text-VLA
onfjects
+XLT
Datas
with the high-gullyy sxoparation piellie.
CM
21
Jsan poaiof
(c) Interleave-VLA Paradigm Overview
objocts
Instrucion
Following
Training
Web
Imsagos
Rest
Crop
Interleave-VLA
Entendable
Gerenilaation
Archilssure
Sketch
Figure I: (a) Our Interleaved X-Embodiment Dataset features diverse, high-quality object-centric images automatically generated from real-world robot demonstrations. (b) Interleave- VLA achieves 2x stronger out-of-domain generalization compared to text-only VLA models in both simulation and real-robot experiments. (c) It enables flexible, zero-shot instruction following with cropped images, web photos, and hand-drawn sketches for practical and intuitive human-robot interaction.
ABSTRACT
The rise of foundation models paves the way for generalist robot policies the physical world. Existing methods relying on text-only instructions often struggle to generalize to unseen scenarios. We argue that interleaved image-text inputs offer richer and less biased context and enable robots to better handle unseen tasks with in-context visual grounding. Building on this insight, we introduce Interleave- VLA, robot learning paradigm extending interleaved image-text in- structions from digital world to directly generating continuous action sequences in the physical world. It offers natural, flexible, and model-agnostic paradigm that extends state-of-the-art assionlanguagg-acttio (VLA) models with minimal modifications while achieving strong zero-shot generalization. Interleave-VL.. VLA also includes an automatic pipeline that converts text instructions from Open X- Embodiment into interleaved image-text instructions, resulting in a large-scale
Emails: (yangxue-20) 2012200190ss 9-sjtu, yanjunch Jēs jtu. edu cr, mdëca udc

---

Published as a conference paper at ICLR 2026
real-world interleaved embodied dataset with 210k episodes. Comprehensive evaluation in simulation and real world show that Interleave-Vel VLA offers two major benefits: (1) improves out-of-domain generalization to unseen objects by 2x compared to text input baselines, (2) supports flexible task interfaces and diverse instructions in a zero-shot manner, such as hand-drawn sketches. We attribute Interleave- VLr'ss strong zero-shot capability to the use of instruction im- ages, which effectively mitigate hallucinations, and the inclusion of heterogeneous multimodal datasets, enriched with Internet-sourcet images, offering potential for scalability. Our website has more information.
INTRODUCTION
The remarkable success of large language models (LLMs) (Achiam et al., 2023 Touvron et al. 2023 Liu et al. 2024a) and vision-language models (VLMs) Baii et al., 2025 2023a Chen et al.) 2024a, Luo et al.) 2025a) has established the paradigm of foundation models in the digital world, which are capable of generalizing across a wide range of tasks and domains. Inspired by this progress, the robotic community is actively developing robotic foundation models (Brohan et al.) 20233] Kim et al. [024] O' et al., (2024) Black et al. [024] In-
2023
Bai et al.
Team 2024| Liu et al.
telligence et al. |025| Chi et al. (023) bring similar generalizability to unseen tasks and scenarios into the physically embodied world. Despite these advances, effective -f--mann generalization of robotic policies remains a key challenge. We argue that the predominant reliance on text-only instructions in current generalist robotic policies constrains their ability to generalize. Text instruc- tions often prove ambiguous or cumbersome in scenarios where users need to specify goals like "pick up object like this," referring to uniquely shaped or colored item. In contrast, inter- leaved image-text instructions allow robots to interpret unseen tasks more effectively by providing in-context visual and textual cues, beyond what text instructions alone can convey.
To develop a general and practical robot policy capable of acting on interleaved image- text instruc- tions in the real world, straightforward solution is to build upon VLA (Kim al.] [024 O' O'Neill
et al. 2024 Brohan et al. 2022 [023 Black et al. 2024] Team et al. 2025) models, which naturally extend VLMs by incorporating action understanding and generation, making them well-suited for robotic tasks. However, current VLA models (Brohan et al.) 2023] Kim et al. 2024 Black et al., 2024) remain predominantly trained on text-only instructions setting we refer to as the Text- VLA paradigm. This limits their ability to benefit from multimodal instruction signals, which have been shown to enhance generalization in vision-language learning (Achiam et al. 2023 Team et al.) 2025). While VIMA (Jian et al.) 2023) served as a conceptual pioneer for multimodal robotics, was restricted to high-level planning in stylized 2D simulations, without further exploring the practicality and generalizability of interleaved instructions. Therefore, there is an urgent need to systematically investigate the potential benefits of training and testing on interleaved instructions over the text inputs that virtually all modern VLAs still adhere to.
To address this limitation, we first build a high-quality interleaved image-text datasets, crucial for training multimodal models. In order to bridge the gap of the lack of image-text interleaved datasets in robotic manipulation, we develop a pipeline to automatically construct interleaved instructions from existing datasets. The proposed pipeline enables automatic and accurate generation of inter- leaved instructions from real-world dataset Open X-Embodiment OO'Neill et al.) 2024]. The result- ing interleaved dataset contains over 210k episodes and 13 million frames, making a large-scale, real-world interleaved embodied dataset. This enables training Interleave-VEL VLA with real-world in- teraction data and diverse visual instruction types.
We then propose new paradigm called Interlcave-VLA. designed for generating continuous ac- tions from interleaved inputs. As illustrated in Figure 2 Interleave- VL. consists of three key com- ponents: (1) lightweight adaptation module that introduces special separator tokens into the to- kenizer, enabling existing VLAs to process interleaved inputs without architectural changes, (2) a scalable training pipeline that leverages large-scale interleaved embodied datasets while preserving original objectives and hyperparameters, and (3) a versatile inference interface that supports both text-only and interleaved instructions, allowing the use of real-world camera crops, web images, or even sketches at test time. This effective design unlocks multimodal instruction-followii capa- bilities for ofate-of-the-art VLAs in an and can be readily extended to other VL. models, such as
2

---

Published as a conference paper at ICLR 2026
OpenVLA (Km et al. 2024). Experimental results demonstrate that Interleave- VLA significantly surpasses text-only baselines in out -ot-a-nn tasks. The interleaved format enables robust zero- shot generalization to novel objects and user-provided sketches unseen during training.
We further investigate the factors behind Interleave- VLA's superior zero-shot performance rela- tive to VLA. We find that both the scale and heterogeneity of the Interleaved X- K-Embodiment Dataset contribute to consistent gains in both low- and high-data regimes. We also categorize three recurring forms of attentional hallucination in Tex--VL: attentional bias, diffused attention, and attention leakage, which arise from linguistic ambiguity and distributional biases in text-only in- struction interpretation. We summarize our key takeaways below:
Generalization failures in VLAs often stem from attentional hallucinations, which we summa- rized as attentional bias, diffused attention, and attention leakage, driven by (1) ambiguous con- texts and (2) training distribution biases (Section 4.1
Interleaved image-text instructions mitigate the hallucination caused by ambiguous contexts, pro- viding less-biased in-context visual grounding for better generalization (Figure 5 in Section 4.1
Modality diversity (e.g, interleaved data, web data) alleviates the hallucination from training distribution biases, further enhancing generalization. Cross-modal training benefits performance in both interleaved evaluation and text-only evaluation (Section 4.3.3
Our core contribution is threefold. (1) We introduce Interleave- -oe:: lightweight, transferable paradigm that enhances the generalization capability of current text input VLA models with inter- leaved image-text instructions. Through comprehensive evaluation, we demonstrate 2x gains in out-of-domain generalization to novel objects, along with emergent zero-shot capabilities for inter- preting diverse visual instructions, such as hand-drawn sketches. (2) We opensource a large-scale, real-world interleaved embodied dataset with 210k episodes and 13 million frames based on Open X-Embodiment, generated by a fully automated pipeline. (3) We provide insights into Interleave- VLA's effectiveness in mitigating attentional hallucinations commonly observed in Text- VLA.
RELATED WORK
Interleaved Vision-Language Models. In the digital domain, recent advances in vision-language al.
models have evolved from handling simple image-text pairs (Liu et al. 2023a Radford et al. 2021
Li ct 2023
Fang
2023)
to processing arbitrarily interleaved sequences of images and Chen et al. 2024a Luo et al.) 2025a Xue et al. 2024: Li et al.)
text (Bai et al.
2025
Team 2024
Alayrac et al.
Chen et al. 2024b Diang al.). This interleaved format allows models to leverage large- scale multimodal web corpora such as news articles and blogs-whert whhre images and text naturally appear in mixed sequences. Such models have demonstrated improved flexibility and generalization, enabling transfer across diverse tasks and modalities (Li et al.). Despite these successes in the digital world, robotic foundation models the physical world have yet to fully exploit the benefits of interleaved image-text instructions. Motivated by the progress of interleaved VLMs, we extend this paradigm to the action modality, enabling loooo-llnnagg-attioonn models to process interleaved instructions. Our results show that multimodal learning with interleaved inputs greatly boosts generalization and displays emergent capabilities in robotic manipulation tasks.
2022,
Vision Language Action Models. Vision-language-acti action (VLA) models have advanced robotic manipulation by enabling policies conditioned on both visual observations and language instruc-
tions (Kim al. 2024 Team al. [2025] Brohan al. 2022
2023 Black et al., 2024 Fang al.)
2025a Wen et al. 2025 Bjorck et al. 2025 22255|Inteliigenee et al.
2025) Most prior VLA models pro-
cess single (Kim al.
(024) or multiple (Brohan al. 2023] Black et al. 2024 observation images
with text-only instructions, with some exploring additional modalities such as 3D (Zhen et al. 2024)
and audio (Zhao et al. 2025b). Only few existing works, such as VIMA (Jiang et al.] 2023). explore the use of interleaved instructions in robotics, evaluating vision-language planning tasks within a high-level 2D action space in simulation. However, they have not investigated the broader bene- fits of interleaved image-text instructions, such (1) their advantages over text-only instructions and (2) their applicability to real-world scenarios involving low-level robotic actions. As a result, the practical value of this paradigm remains underexplored due to a lack of real-world datasets and policies capable of handling such input In this work, we make the first step to bridge this gap by proposing Interleave-VL.C VL.A: a simple, model-agnostic paradigm that extends existing VLA models to
3

---

Published as conference paper at ICLR 2026
Method
Plug-in
Multimodal Instructions
Backbone Agnostic
Data Source
Simulation / Physies Engine
Auto Angmeeatation
Customized Image
Instruction
Gemini Roboties Tean
2025
X
GROTT NI
CoT- VL 2an
X
X
V
Helix
(Cu Br 8133
X
/
X
/
ReBbot
A425b
/
V
V
Being-Hin
HD Lavelal
X
V1.A8
Zhivzr
MP.30
x
X
X
NaVIL
Chaig et 2125
X
7
NORA
[Hain
2125
Interleav-VLA.. (Ours)
Table 1: Comparing Interleave-VLL. with representative VLA methods. Unlike prior systems that depend on fixed backbones, source external Internet or simulation data, and accept only text inputs, Interleave- VLA operates backhone-agnostic plug-in that supports image-text instruc- tions. reuses existing robot datasets without relying on external data acquisition, provides au- tomatic, scalable data augmentation, and uniquely enables custom image-conditioned instructions (like sketches) at test time, yielding more versatile, practical and generalizable VLA paradigm.
support interleaved image-text instructions with minimal modifications. Our comprehensive exper- iments demonstrate that interleaved instructions substantially improve generalization, and unlock strong zero-shot capabilities for diverse user- seer-provided inputs. This highlights the practicality and scalability of interleaved image-text instructions for real-world robotic manipulation.
INTERLEAVE- VLA AND OPEN INTERLEAVED X- X-EMBODIMENT DATASET
3.1 PROBLEM FORMULATION
A discrepancy exists between the input modalities of modern Vision-Languaae Models (VLMss (Alayrac et al. 2022 Bai et al. 2025 Team 2024 Xue et al. 2024). which accept arbitrar- ily interleaved inputs, and most LagionLLnguage-- Action (VLA) models, which typically operate on a single text instruction. We formally define this text-only instruction paradigm as Text-VLA In this work, propose Interleave- VLA, generalized paradigm that allows a robotic policy generate actions conditioned on interleaved image-text inputs. This formulation elevates VLA models to the same imput flexibility as VLMs, thereby rendering Eext-LLA specialized instance.
Formally. policy under the Interlcave- VLA paradigm generates an action at each timestep t by sampling from distribution conditioned on the state 8, 0₃ To(- 8t) where the state is defined as tuple = (Je, q:. T). This tuple comprises the current visual observation I (e.g., image or set images), the robot's proprioceptive state qr, and an interleaved instruction sequence I. The ordered instruction sequence is represented as I (u1,....
... U.) ), where each token új E Uix ing belongs either to the set of text tokens Viea or to the set of image tokens Ving- Notice that Interleave-
VLA can degenerate to special case of standard Text- VLA when all uj Vieu-
We show a typical example of standard text-only instruction and interleaved instruction:
Text-only: <obs> Place [the blue spoon near microwave] into [silver pot on towel].
Interleaved image-text: <obs> Place [imagel into [image2
where <obs> stands for observation image(s), and [imagel ] and [image2 are in- terleaved instruction images. As shown in Figure Interleave- VLA supports more flexible formats.
3.2 INTERLEAVE-VLE PARADIGM
Outlined in Figure 2 our Interleave-VEr VLA paradigm, designed for generating continuous actions in the real world from interleaved inputs, comprises three core components: (1) straightforward and effective adaptation module, (2) scalable training process tailored for interleaved data, and (3) versatile inference interface that supports interleaved instructions.
Adaptation. Interleave- VLA extends Text- VLA by introducing special separator tokens into the tokenizer of the base VLA model, allowing to distinguish between image and text tokens. The input processor is updated support the interleaved format, while the core VLA architecture

---

Published as a conference paper at ICLR 2026
Training
Interleave- VLA
Inference
Crop
Adaptation
Skaithh Grap this Tor me
of
Interes
Internel
Pre-trained
Action Expert
X-Embodiment
Intericaved
Dataset
Move redbull
brown cltp.
Text
-D01 insael
1812
Crop
blue stripeal
Figure 2: Overview of the Interleave-VLLe paradigm, featuring an extendable adaptation of Text- VLA to handle interleaved inputs, scalable training on a constructed large interleaved dataset, and versatile inference that supports a wide range of interleaved instructions.
OWLvv2
Utensils
Cleaning
O pen Interleaved X-E Embodlmeet
Instructoon Put eggplant.on on plate.
Retect plate
Shapes Blocks
Owen2. 5-VL
Dataset
Mick rid
The point eggplant <xls, ylto
Detect eguplont
Containers
Yes. It a plata.
Potnt Drivan
Segmestatius
Open Vocabulary Delection
Kitchenware
OO SAM 2
Data Quality Verification
Figure 3: Left: Our open interleaved X- Embodiment dataset features large number of high- sugh-quality cropped images with diversity across objects. Right: Interleave dataset generation pipeline: (1) In- struction Parsing: use LLM to extract key objects from language instructions. (2) Open-VVoabulle Detection: use OWLv2 to locate and crop target objects from trajectory frames based on the parsed instruction keywords. (3) Data Quality Verification: use Qwen2.5-VL to verify the detected objects, and if needed, provide keypoints for more precise segmentation using Segment Anything.
mains unchanged. This paper focuses on applying Interleave-VLA to To (Black et al., 2024), a state-of-the-arr TeXt-VLA model. Despite its pretrained Paligemma (Beyer et al.) (2024) lacking na- tive support for interleaved data, Interleave- VLA enables this capability. Our adaptation is effective in enhancing the zero-shot generalization potential of To. Moreover, the simplicity of this adaptator makes it easily applicable to other VLA models, such as Open (Kim et al. (024). For further details on our model-agnostic adaptation, see Appendix
model using the large-scale interleaved embodied dataset from Section 3.3, without modifying its hyperparameters or flow matching objective. The training process efficiently scales with the dataset size and cross-modal instruction diversity.
Training. We train the inteleeved-addaper
Inference. Interleave- VLA paradigm supports both text and interleaved instructions during infer- ence, with interleaved inputs significantly improving generalization to unseen scenarios. Interleaved images in prompts offer great versatility, as they can sourced from diverse sources such as robot camera crops, web images, or hand-drawn sketches, even when the image styles differ from those in the robot's training data. To simplify interaction with the robot, we also design a user-friendl. GUI.
3.3 CONSTRUCTION OF OPEN INTERLEAVED X- X-EMBODIMENT DATASET
A large-scale and high-quality pretraining dataset scales up iision-language Language-action action (VLA) (Black al. 2024 Brohan et al., 2023 Kim et al. 2024) training. However, current real-world datasets only include text instructions and thus do not support Interleave VLA training. We consequently design unified pipeline to automatically relabel and generate interleaved data across diverse datasets.
Our overall dataset generation pipeline consists of three main steps: instruction parsing, open- vocabulary detection, and data quality verification, illustrated in Figure 3 First, we use Qwen2.5 (Yang et al.] (024) to extract key objects from language instructions. Unlike rule-based NLP tools like SPaCy (Honnibal 2017), Qwen2.5 adapts diverse instruction formats without requiring case-specific rules. also effectively summarizes lengthy instructions, such as those in datasets like (Shah et al. 2023). Second, for open- detection, we the open-vocabulary
5

---

Published as a conference paper at ICLR 2026
In Domaln Tasks
Out-of- Tasks
pepper Coc
spoion
boon !
60
parsa SUrVer
spatib
black spondaa
Figure 4: Left: Illustration of generalization settings in SIMPLER. (a) Visual generalization: unseen environments, tablecloths, and lighting conditions. (b) Semantic generalization with novel objects from known categories. (c) Semantic generalization with objects from entirely categories not seen during training. Right: Real-world generalization experiments. In-Domain and out-of-Domani settings in the real world FANUC LRMate 200iD/7L robotic arm.
detector OWLv2 (Minderer et al.) 2023) to locate and crop target objects from trajectory frames based on the parsed instruction keywords, achieving 82.6% accuracy. Finally, we introduce data quality verification for harder cases where OWLv2 fails: Qwen2.5-VL. (Bai et al. 2025 verifies the detected objects, and if needed, provides keypoints for more precise segmentation using Segment Anything (Ravi al. 2024). This collaborative approach leverages the complementary strengths of VLMs, raising accuracy to 95.6%. Detailed metrics and analysís are provided Appendix
We release large-scale interleaved cross-embodiment dataset in the real world, featuring diverse tasks and instructions. This dataset integrates 11 datasets from Open X-Embodiment (O'Neill et al.,
2024), including RT-1 (Brohan et al. 2022). Berkeley Autolab UR5 (Chen al.), IAMLab CMU
Pickup Insert (Saxena et al. 2023), Stanford Hydra (Belkhale et al. 2023), UTAustin Sirius (Liu
al.
2023b), Bridge (Walke al. 2023a Jaco Play (Dass al. (002)). UCSD Kitchen Yan al.
2023
BC-Z Jang et al. 2022). Language Table (Lynch et al. 2023) and UTAuutin Mutex (Shah
al.
2023). The curated dataset comprises 210k episodes and 13 million frames, spanning 3,500
unique objects and wide range of task types. To enhance instruction diversity, we augment our dataset by randomly integrating Internet-sourced images alongside the original text instructions.
EXPERIMENTS
In the experiments, our aim is to answer the following research questions:
(1) How does our Interleave- VVn paradigm compare to conventional Text-VLa paradigm?
(2) What are the common failure modes of Text- VLA, and how does Interleave- VLA address them?
(3) What are the benefits the different design choices in our Interleave- paradigm?
4.1 SIMULATION COMPARISON WITH TEXT-VLA
Task setup. We use SimplerEnv (Li
2024 real-to-sim evaluation suite, efficiently evalu- ate policies in realistic scenarios. Performances are tested on SimplerEnv-Bricee -Bridge setup. which uses WidowX robot configuration compatible with the BridgeData V2 (Walke et al. (023b). Since Sim- plerEny is built for VLA, to enable scalable evaluation of VI. models, we extend SimplerEnv with interleaved image-text prompts automated pipeline (Section 3.3).
In addition to the four HapplrrEny-Brideee tasks from BridgeData V2, we include ten new tasks for generalization evaluation. Following commonly used Stone et al. (2023). these tasks focus on assessing both visual generalization and semantic generalization. Visual generalization assesses robustness to novel foreground, lighting, and environment backgrounds. Semantic generalization assesses the model's ability to manipulate in novel scenarios and the presence of diverse distractors. This evaluation further divided into two sub-categories: (1) novel objects from previously seen categories, and (2) objects from entirely unseen categories. See left part of Figure for overview.

---

Published as a conference paper at ICLR 2026
Table 2: Interleave-VLA and LLX comparison on SimplerEnv. In-Domain includes tasks following SimplerEn--Bridge setup. We add 3 Out-of-Domain evaluation suites, namely: Visual, Novel Object, and Novel Category. To with full adaptation (Interleave- VLA Full) performs better than roo with no adaptation (Text-VLA) by over 2 in out-of-domain semantic generalization tasks. It also outperforms T0.5 which enjoys additional pretraining with additonal object grounding and detection VQA data. Results are evaluated on 3 seeds. We use bold and underline to represent
the st and nnd highest. For quantitative breakdown of failure modes, please refer to Figure 11
Model
Paradigm
Train/Kival
In In-Domain
Out-of-Demain
Modality
Visual
Novel Object
Novel Category
Avg.
RT-1-X (y'Nelll al 2424
Text-V VLA
Texffext
0.5
0.0 +
0.5
5.8 5.3+003
3.2
Octo
Team 8 K124
Test- VLA
Text/Text
17.4 1.3
12.5 + 0.1
10.8 -
8,2 0.2
10.5 +
Spati
2025
Test-V
Tea//Tsi
17.1
17.6 1::64:00:
18.0
Intellineke B
Tex-VV A
Text/Text
4-.9
53.9+1.11
0.3
41.8
40.0
Biak ar 272
Text-Vi VLA
Text/Tent
68.1 +
72.4 1.1
26.0 = 3.5
19.3 + 1.5
30.5 + 0.3
Black
2024
Interlavee VLA
(Pantal)
Interloww//Tsst
20.1
76.8 + 0.2
35.8
20.9 +
43.6 +
Black ct
2024
Interleave.
(Full)
Intereane/lntateeee
70.5 12 1.3
73.2 4. 0.3
53.8 1: 1.5
573
60.6 1.1
Baselines. We evaluate Interleave- VLA against leading Text-VLA models, including RT-1-X (Bro- han et al. 2022), Octo (Team et al., 2024), and SoTA wo (Black et al. (024)). To directly compare with Text- VLA, implement Interleave- VLA (Partial) on mo. which is trained with interleaved inputs but tested with text-only instructions. Finally, we evaluate the full potential of the Interleave- VLA paradigm, where To is both trained and tested using interleaved inputs. Both Interleave-VLA and TeXt-VLA paradigms are trained on trajectories from BridgeData V2 WWlke et al., 2023a).
Results. In-domain results in Table 2 show that our Interleave- VLA paradigm performs on par with VLA for familiar tasks, demonstrating that interleaved instructions are interpretable thanks to Interleave- VLA training process. For out-of-domain tasks, Interleave- VLLA (Partial) already outper- forms VLX, benefiting from the multimodal nature of interleaved data, which helps mitigate overfitting. The full Interleave-Ve VI.A further enhances generalization, achieving 2x better perfor- mance on semantically out-of-domain tasks.
Analysis. The substantial performance gains of Interleave-VLA (Full) over TeX and Interleave- VLA (Partial) mainly stem from the explicit visual grounding supplied by interleaved instruction images, which reduces a phenomenon we term attentional hallucination. To qualita- tively illustrate this, compute the attention scores of target object tokens relative to the tokenized observation in out-of-domain settings. As shown in Figure S we identify three primary failure pat- terns: (1) Attentional Biax, where focus is incorrectly allocated to prominent distractor instead of the target object; (2) Diffused Attention, characterized by a complete lack of a focal point as atten- tion spreads thinly across the entire scene, suggesting model uncertainty; and (3) Attention Leakage, where the model correctly identifies the target but its focus is not tightly contained, scattering onto irrelevant background areas. These failures can be attributed to semantic ambiguity in cluttered vi- sual contexts and distributional bias in the training data. For example, semantic ambiguity arises when the instruction says "toy dinosaur" but a similarly shaped toy elephant is present, the text-only Text-VLA model often makes an arbitrary choice; distributional bias manifests when Text-VLA misidentifies a Coca-Cola can as a Red Bull can because the rare token "redbull" segmented into "red" + "bull", causing it to over-attend to "red" and biasing attention toward the familiar red Coca-Cola can. These built-in biases are difficult to address with conventional Text- VLA. In con- trast, Interleave- LLAA outperforms Text- VLA baselines by leveraging in-context visual grounding and mross-modality training to reduce attentional hallucinations.
4.2 REAL ROBOT COMPARISON
Task setup. We evaluate on a FANUC LRMate 200ID/7L. robotic arm equipped with an SMC gripper. Two manipulation tasks considered: (1) picking up food or fruits, and (2) picking and placing kitchenware. To assess semantic generalization, we follow the SimplerEnv setup. See the right part of Figure for overview, with additional details in Appendix G.3
Baselines. All baselines use the same base VLA model Ny, with two following the Interleave- VLA paradigm and one using Text-VLA, Each baseline is finetuned on 20 teleoperated demonstrations per object, collected using a space mouse. Optionally, pretraining is performed on the robot data in Section 3.3 training Text-VL.A VI.A on text instructions and Interleave-VL. on interleaved instructions.
7

---

Published as a conference paper at ICLR 2026
Zucchini
Stapler
Tennis
Redhull
Toy
Interleave-VLL Text-VLA
Attention Leakage
Diffused Attention
Attentional Bias
Figure 5: Qualitative analysis of Interleave-VLA'S improved performance over the Text- VLA paradigm. In out-of-domain SimplerEnv tasks with unfamiliar objects, Text-VLA displays at- tentional hallucination, which typically manifests in three categories: (1) Attention Leakage: the target is partially attended, but focus spills onto irrelevant background distractor regions; (2) Dif- fused Attention: attention is broadly scattered with no dominant focus, indicating uncertainty about the target; (3) Attentional Bias: attention centers
salient distractor instead of the true target. Interleave- VLA effectively mitigates these issues by leveraging in-context visual cues from inter- leaved instructions, demonstrating consistent attention target objects.
Table 3: Comparison of success rates (Succ) and correct object picking rates (Acc) in real-robot experiments. All the baselines use the base VLA model To. Interleave- VLA adapted achieves 2- 3x higher out-of-domain performance compared to Text-VLA. "PT" indicates pretraining on our interleaved dataset built Section 3.3 Notably, although the pretraining dataset does not include FANUC robot arm data, it still enables strong cross cross-embodiment transfer to FANUC.
In-Domain
Out-of-Domain
Paradigm
pepper
corn
cup
Avg
bean
lemon
Avg
Sace:
Acc.
Suce.
Acc.
Succ.
Acc.
Suce.
Arc.
Suce.
Acc.
Sacc.
Acc.
Suce.
Acc.
Interleave- VLA
17
33
0
33
0
33
33
0
40
0
33
37
Text-VLA
58
33
100
100
94
42
Balerleav-VLLA
58
100
75
100
67
100
67
100
75
100
67
100
71
100
pasta server
spoon
knife
Avg
spatula
black spatula
Avg
Sucee
Suce.
Acc.
Succ.
Acc.
Suce.
Acc.
Suce.
Acc.
Suce.
Acc.
Suce.
Acc.
Interleave VLA
33
8
58
17
58
19
61
67
50
59
Text-VLA
$8
83
58
75
33
58
50
72
8
33
42
21
25
Interleave VLA
58
100
Results. Table shows that Interleave- VLA achieves 2-3x higher out-of-domain performance com- pared to Text- VLA when both are pretrained. Unlike the SimplerEnv experiments, where large-scale BrtageData V2 supports strong performance, the real-robot setup relies on a smaller self-collected dataset. In this low-data regime, directly finetuning To performs poorly (first row). Pretraining on the Interleaved X-Embodiment dataset significantly boosts performance through effective cross- embodiment transfer, reducing the need for laborious data collection.
4.3 ANALYSIS OF INTERLEAVE-VLAN GENERALIZATION AND EMERGENT CAPABILITIES
4.3.1 INTERLEAVE-VLA ADAPTATION: EXTENDING TO OTHER VLA MODELS
The model-agnostic design of Interleave-VIL allows easy adaptation to other VLA models, demon- strating effectiveness enhancing manipulation generalization across diverse architectures. To validate this, we extend Interleave- VLA to OLe (Kim et al.] 2024), a state-of-the-an VLA model with distinct architecture and training objective compared to No- We evaluate it on VIMA- Bench (Tiang al.] 2023), which includes four levels of manipulation planning tasks involving ob- jects with irregular, cartoon-like shapes and textures. As shown in Figure we compare Interleave- VLA against several end-to-end baselines. including the Text-VLA model VLe and other VIMA-like models such as VIMA-Gato, VIMA- AMMA-Flamingo, and VIMA-GPT (Jiang et al.) 2023). Across all four generalization levels, general Interleave-Ve VLA paradigm, when directly extended to Open achieves the best performance without relying on any taec-fpcc designs. Details the adaptation and evaluation are provided in Appendices D.2 and G.2
8

---

Published as a conference paper at ICLR 2026
Table 4: Interleave-VLA unlocks powerful zero-shot generalization diverse instruction modali- ties, including hand-drawn sketches, user-cropped images, and Internet photos, without ever seeing them in training dataset. The consistently high accuracy demonstrates that Interleave- VLA can ro- bustly interpret and execute visually grounded instructions, showing strong potential for flexible and practical human-robot interaction. For more styles of sketches and potential failure modes, please
refer to Table 8 and 9 in Apppendix C
Task
Prompt
A Succ. (%)
A Acc. (%)
Prompt B
Suce. (%)
B Acc. (%)
58.3
90.0
48.8
86.0
75.8
100
58.8
100
71.7
100
80.8
100
70.0
96.0
73.3
100
69.6
100
76.3
75.5
100
71.7
100
4.3.2 INTERLEAVE-VLA INFERENCE: FLEXIBILITY AND EMERGENT GENERALIZATION
The inference interface of Interleave- VLA, which we show effectively reduces attentional halluci- nation problem, is highly versatile. Interleave- VIA demonstrates strong performance across diverse ways of specifying instructions VIMA-Bench, including goal image matching and multi-image instruction following. These results demonstrate the flexibility of the Interleave-VLA paradigm, driven by its unified image-texi interleaved instructions for general robotic manipulation.
Building on its versatile inference interface, Interleave-VLA further showcases an emergent capa- bility to interpret instructions a completely zero-shot manner, directly handling unseen input modalities without any additional finetuning. Table 4 demonstrates the examples of image instruc- tion types and their corresponding high performance. Instructions can be in diverse formats, includ- ing: (1) Cropped Image Instructions: Users directly crop region from the screen to indicate the target object. (2) Internet Image Instructions: Users may supply any image- as photo retrieved from the Internet-to represent the desired object. (3) Hand-Drawi Sketch Instructions:
Users can draw sketches or cartoons about their intentions.
The interleaved instruction format naturally accommodates diverse imput types, making human-robot interaction more intuitive by re- moving the need for users to precisely describe complex objectives with detailed text. This flexibility significantly enhances the model's ability to generalize, as evidenced by the im-
VIMA-Bench
100
VMA
Succase Rami PR
60
provements observed in both in-domain and out-of-domain tasks, where interleaved image- text instructions effectively reduce attentional hallucinations VI.A models. These advance- ments in Interleave-VL. collectively pave the way for more adaptable robotic systems.
Figure 6: VIMA-Bench results across four gen- eralization levels: L1 (object placement), L2 (novel combination). 1.3 (novel object), and L4 (novel task). To demonstrate the extendability of our Interleave-VLA paradigm, we apply it to another Text-VLA model, OpenVLA. Interleave- VLA outperforms Text-VLA by 2 across all dif- ficulty levels, further highlighting its superior gen- eralization capabilities with evidence from new task sets and a different base VLA model.
4.3.3 VLA
TRAINING: MODALITY
AND INSTRUCTION DIVERSITY MATTER
The most obvious scaling law of Interleave- VLA dataset size, which shown in large data domain (Appendix HH and low-data do- main (Table 3) Overall, our results underscore the importance of the curated large-scale Interleaved X-Embodiment Dataset (Section 3.3 in fostering robust and generalizable Interleave-VIL In this section, we delve deeper into the training data and identify two key factors that drive scalability and generalization: (1) the modality diversity of the dataset and (2) the diversity of prompt images.
The diversity of modalities in training dataset crucial for achieving robust performance VLAs, particularly for out-of-domain generalization. This principle is empirically demonstrated by com-

---

Published as a conference paper at ICLR 2026
Table 5: Importance of prompt im- age diversity for Enterleave-VLA.
Table 6:
Interleaved instructions contribute through both format and content. Visual-goal cues drive out-of- domain generalization by providing explicit image infor- mation, while the interleaved format offers complementary gains and prevents task ambiguity, especially for underspec- ified goals such as "Move Near". See Section 4.4 for details.
"In-Domain" stands for seen tasks; "Out-of-Domain" reports unseen sce- narios. Combining both task-specific and Internet images as prompts achieves the best overall performance.
Instruction
In-Domain
Unseen IG
Unseen Obj
Unseen Cat.
Move Neur
Data
In-Domain
Out-of-Domain
(train and Inference)
Ouly
99.2
69.1
Text
69.2
71.4
30.2
21.0
66.6
Mixed
Task-specific Only
71.0
67.5
67.1
71.7
Interiorved Img-Test
Visual Goal
67.8
71.3
74.6
73.4
48.0
53.9
54.2
51.9
68.
0.0
paring the performance of Interleave- VLLA (Partial) and TeXI-VLA, which share an identical ar- chitecture (see Table 2). wo trained with cross-modal, interleaved image-text instructions achieves absolute improvements of +2.5 on familiar in-domain tasks and a more substantial +5.7 on tasks requiring generalization to new objects. We attribute this performance gain to the development of richer multimodal representations by mitigating the model's tendency to overfit to unimodal text signals (Alayrac al. 2022).
Instruction image diversity is crucial as well, Table 5 demonstrates that combining Internet images with task-specific images cropped from robot observations yields the best overall performance. Us- ing only Internet images leads to lower in-domain accuracy due to limited task relevance, while re- lying solely on cropped images improves in-domain results but lacks diversity, Mixing both sources provides complementary advantages, resulting in enhanced accuracy and stronger generalization.
4.4 INTERLEAVED INSTRUCTION: BOTH FORMAT AND CONTENT ARE CRUCIAL
The interleaved image text instruction in Interleave- VLA contains two separable factors: (1) the format, which allows image and text tokens to appear in arbitrary order, and (2) the content, which introduces explicit visual goal cues. Both contribute generalization, but in different ways.
To isolate the contribution of the visual goal signal, we perform an ablation in the SimplerEnv- Bridge setting (Table 2 In Table 6 We compare interleaved image-text content ("Put [Object A] to [Object B]"" with a pure visual-goal content ("[Object A][Object BJ"") The first four columns confirm that adding visual goal cues improves performance. However, the interleaved format itself is equally crucial, as highlighted by the "Move Near" column. Under the Visual Goal format, the model consistently misinterprets the task as a "put on" operation. This occurs because many com- mon obbot-instruction templates, e.g.. "Put object A [near / to the left of / on] object B", become ambiguous when expressed only through object images. Since "put on" primitives are far more prevalent than "move near" in the training data, the Visual Goal format tends to collapse to this dominant interpretation. In contrast, the interleaved format provides an unambiguous representation that is necessary for leveraging embodied data across diverse manipulation tasks.
In summary, interleaved content offers image-text grounding, whereas the interleaved format enables task unification with minimal ambiguity.
CONCLUSION
Text-only instructions in most robotic policies can be insufficient for unseen scenarios and even causes hallucinations. To address this, we propose Interleave- VLA, a simple and effective paradigm for adapting existing VLA models to process interleaved image-text instructions. To overcome the lack of real-world interleaved datasets, we develop an automatic pipeline that generates large- scale dataset with 210k episodes and 13 million frames from Open X-Embodiment. With minimal modifications to current VLA models, Interlcave-VE VLL achieves 2x improvement in generalization across both simulation and world experiments. Furthermore, our approach demonstrates strong emergent zero-shot generalization to diverse user instructions never seen during training-including hand-drawn sketches, cropped images, and Internet photos-make it both practical and flexible for real-world robotic applications.
I Limitations. Please refer Appendix B for detailed discussion.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This research adheres to the ICLR ethical guidelines and upholds the principles of responsible re- search, We ensure that no personally identifiable, sensitive, harmful data were used. Our ex- periments were based on publicly available datasets and did not involve any human subjects or vulnerable groups. We have considered the potential societal impact of our methods, including the risk of misuse, and believe that these contributions primarily advance scientific understanding and do not pose foreseeable harm.
REPRODUCIBILITY STATEMENT
We follow the reproducibility guidelines in the ICLR 2026 author guidelines. We will open source code, configuration files, and scripts to reproduce results, including dataset construction, model training, and evaluation, on platforms such as GitHub and Huggingface as soon as possible.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXhv:2303.08774, 2023.
Ban-BBatiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716- 23736, 2022.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar XxIvv22502.13923 2025.
Suneel Belkhale, Yuchen Cui, and Dorsa Sadigh. Hydra: Hybrid robot actions for imitation learning. In 7th Anmual Conference Robot Learning, 2023.
Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. CoRR, 2024.
Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da. Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t nl: An open foundation model for generalist humanoid robots. arXiv preprint 2rXiv:2503.14344 2025.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. no: A vision-language-action flow model for general robot control. arXiv preprint arXiv:24100 20..222 164, 2024.
Kevin Black, Manuel Y. Galliker, and Sergey Levine. Real-time execution of action chunking flow policies, 2025. URL /tRp::tss://aat
org/ abs///as../////////// 07339
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishhan, Karol Hausman, Alex Herzog, Jasmine Hsu, al. Rt-1: Robotics transformer for real-world control at scale. arXiv e-prints, pp. arXiv-2212, 2022.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choroman- ski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision- disioo-language-cctto models transfer web knowledge robotic control. arXiv preprint arXiv:2307. /58/8, 2023.
11

---

Published as a conference paper at ICLR 2026
Lawrence Yunliang Chen, Simeon Adebola, and Ken Goldberg. Berkeley URS demonstration dataset.
5/home
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024a.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024b.
An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou, Jan Kautz, Erdem Bıysk, Hongxu Yin, Sifei Liu, and Xiaolong Wang. Navila: Legged robot vision-language-acioon model for navigation, 2025, URL
04453
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, pp. 02783649241273668. 2023.
Can Cui, Pengxiang Ding, Wenxuan Song. Shuanghao Bai, Xinyang Tong, Zirui Ge, Runze Suo, Wanqi Zhou, Yang Liu, Bofang Jia, Han Zhao, Siteng Huang, and Donglin Wang. Openhelix: A short survey, empirical analysis, and open-source dual- vla model for robotic manipulation, 2025. /
/tatps://arxiv..r. www.xxiv..gg/aa... abs
03912
Shivin Dass, Jullian Yapeter, Jesse Zhang, Jiahui Zhang, Karl Pertsch, Stefanos Nikolaidis, and cowp://acccccm/llaac/////cc
Joseph J. Lim. Clvr jaco play dataset, 2023.
Shengliang Deng, Mi Yan, Songlin Wei, Haixin Ma, Yuxin Yang. Jiayi Chen, Zhiqi Zhang, Taoyu Yang. Xuheng Zhang, Wenhao Zhang, Heming Cui, Zhizheng Zhang, and He Wang. Graspvla: grasping foundation model trained on billion-scale synthetic action data, 2025. htLhttps:
23233
Yu Fang, Yue Yang. Xinghao Zhu, Kaiyuan Zheng, Gedas Bertasius, Daniel Szafir, and Mingyu Ding. Rebot: Scaling robot learning with real-ta-sim-to-real robotic video synthesis. arXiv preprint arXiv:2503. 14526, 2025a.
Yu Fang, Yue Yang. Xinghao Zhu, Kaiyuan Zheng, Gedas Bertasius, Daniel Szafir, and Mingyu Ding. Rebot: Scaling robot learning with coalto-lim-t---aal robotic video synthesis, 2025b. 14526
httpstts::/// /ttsxv///xvvv..//.
Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19358 19358-19369, 2023.
Ankit Goyal, Hugo Hadfield, Xuning Yang, Valts Blukis, and Fabio Ramos. Vla-O: Building state- of-the-art vlas with zero modification. arXiv preprint arXiv:2510.1300 2025.
Jiayuan Gu, Sean Kirmani, Paul Wohlhart, Yao Lu, Montserrat Gonzalez Arenas, Kanishka Rao, Wenhao Yu, Chuyuan Fu, Keerthana Gopalakrishnan, Zhuo Xu, et al. Rt-trajectory: Robotic task generalization via hindsight trajectory sketches. In The Twelfth International Conference on Learning Representations.
Matthew Honnibal. spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. (No Title), 2017.
Yihan Hu, Jiazhi Yang. Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tian- wei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia. Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning oriented autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17853-17862. 2023.
Sanning-oriented
12

---

Published as a conference paper at ICLR 2026
Jialei Huang, Shuo Wang, Fanqi Lin, Yihang Hu, Chuan Wen, and Yang Gao. Tactile-vla: Unlock- ing vision-language-aatiin model's ' physical knowledge for tactile generalization, 2025. URL
https://arxi:: org/ abs 250. 09160
Chia-Yu Yu Hung, Qi Sun, Pengfei Hong. Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, and Soujanya Poria. Nora: A small open open-sourced generalist vision language action model for embodied tasks, 2025. URL htt.httts:::aiiiii wwwpxiii.orgggbooo / /aarg//bb//5044 19854
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. To.s: rision-lagguage-attiin model with open-world generalization. arXiv preprini arXiv:2504. 16054, 2025.
Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. Be-z: Zero-shot task generalization with robotic imitation learning. In Confer- ence on Robot Learning, pp. 991-1992 PMLR, 2022.
Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. In Advances in Neu- ral Information Processing Systems (NeurIPS), NIPS '24. Curran Associates Inc., 2024. ISBN 9798331314385
Xiaosong Jia, Yihang Sun, Junqi You, Songbur Wong, Zichen Zou, Junchi Yan, Zuxuan Wu, and Yu- Gang Jiang. Efficient-lvsm: Faster, cheaper, and better large view synthesis model via decoupled co-refinement attention. International Conference Learning Representations (ICLR), 2026.
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. Transactions on Machine Learning Research.
Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yonggiang Dou, Yanjun Chen, Li Fei- Fei, Anima Anandkumar, Yuke Zhu, and Linxi Fan. Vima: Robot manipulation with multimodal prompts, 2023.
Yang Jin, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, CHEN Bin, Chengru Song, Di ZHANG, Wenwu Ou, et al. Unified language-vision pretraining in I1m with dynamic discrete visual tokenization. In The Twelfth International Conference on Learning Representations.
Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Iharz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforce- ment learning for vision-based robotic manipulation. In Conference on robot learning, pp. 651- 673. PMLR, 2018
Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. In Forty-first International Conference on Machine Learning. 2024.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, et al. Openvla: An open-source vision-language-action model. In 8th Annual Conference on Robot Learning, 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Opti- mizing speed and success. arXiv preprint arXiv:2502. /9645, 2025.
Feng Li. Renrui Zhang, Hao Zhang, Yuanhan Zhang. Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. dava-interleave: Tackling multi-image, video, and 3d in large multimodal models. In The Thirteenth International Conference on Learning Representations.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
Qifeng Li. Xiaosong Jia, Shaobo Wang, and Junchi Yan. Think2drive: Efficient reinforcement learning by thinking with latent world model for autonomous driving (in carla-v2). In European Conference on Computer Vision (ECCV), pp. 142-158. Springer, 2025.
13

---

Published as a conference paper at ICLR 2026
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sich, Sean Kirmani, et al. Evaluating real-world robot manipulation policies in simulation. In 8th Annual Conference on Robot Learning, 2024.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 2024a.
Haotian Liu, Chumyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36:34892-34916, 2023a.
Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, and Yuke Zhu. Robot learning the job: Human loop autonomy and learning during deployment. In Robotics: Science and Systems (RSS). 2023b.
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt- lb: diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07888 2024b.
Gen Luo, Xue Yang. Wenhan Dou, Zhaokai Wang, Jiawen Liu, Jifeng Dai, Yu Qiao, and Xizhou Zhu. Mono-internvl: Pushing the boundaries of monolithic multimodal large language models with endogenous visual pre-training. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2025a.
Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-hO: Vision-languuag--ctionn pretraining from large- scale human videos, 2025b. URL https: /tttpp://aaxivv. wttpl//..rgaabb//////a 2557711597777777
Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023.
Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey vision- language-action models for embodied ai, 2025.
Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling opan-vocabulary object detection.
Advances in Neural Information Processing Systems, 36.72983-73007, 2023.
Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 6892 6892-6903. IEEE, 2024.
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding. Zhigang Wang. Jia Yuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual- language-action model. arXiv preprint arXiv:2501.15830 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 9748-8763. PmLR, 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Time Maan-Tingun Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.000 14, 2024.
Saumya Saxena, Mohit Sharma, and Oliver Kroemer. Multi-resolution sensing for real-time control with vision-language models. In 7th Annual Conference on Robot Learning, 2023.
Rutav Shah, Roberto Martín-Martin and Yuke Zhu. Mutex: Learning unified policies from multimodal task specifications. In 7th Annual Conference on Robot Learning. 2023. URL
httpss://ppenrewe net wwwp:/teeeoootiteete.... qaaEz
14

---

Published as a conference paper at ICLR 2026
Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, and Chelsea Finn. Yell at your robot: Improving on-the-fft fly from language corrections. CoRR, 2024.
Lucy Xiaoyang Shi, Michael Robert Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tan- ner, Anna Walling. Haohuan Wang, Niccolo Fusai, Adrian Li-Bell, et al. Hi robot: Open-ended instruction following with hierarchical vision- loooo--lnuugg-acee tional Conference on Machine Learning, 2025.
models. In Forty-second Interna-
Austin Stone, Ted Xiao, Yao Lu, Keerthana Gopalakrishnan, Kuang-Huei Lee, Quan Vuong, Paul Wohlhart, Sean Kirmani, Brianna Zitkovich, Fei Xia, et al. Open-world object manipulation using pre-trained vision-language models. In 7th Anmual Conference Robot Learning, 2023.
Priya Sundaresan, Quan Vuong, Jiayuan Gu, Peng Xu, Ted Xiao, Sean Kirmani, Tianhe Yu, Michael Stark, Ajinkya Jain, Karol Hausman, et al. Ri-sketch: Goal-conditioned imitation learning from hand-drawn sketches. In 8th Anmual Conference on Robot Learning. 2024.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint
arXiv:2405.09818, 2024.
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslic, Jean-Baptiste Alayrac, Montser- rat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20020 2025.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: open-source generalist robot policy. arXiv preprint arXiv:2405. 12213, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. Fastvit: A fast hybrid vision transformer using structural reparameterization, 2023. URL https://
http://rggbbb//00/00 14189
Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, Vivek Myers, Kuan Fang, Chelsea Finn, and Sergey Levine. Bridgedata v2: A dataset for robot learning at scale. In 7th Annual Conference on Robot Learning, 2023a.
Homer Rich Walke, Kevin Black, Tony Z. Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning scale. In Conference on Robot Learning, pp. 1723-1736. PMLR, 2023b.
Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Zhibin Tang, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, et al. Tinyvla: Towards fast, data-efficient vision-languagg-action models for robotic manipulation. IEEE Robotics and Automation Letters, 2025.
Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael Ryoo, et al. xgen-mm (blip-3): A family of open large multimodal models. arXiv preprint 2488::2408..8872.. 2024.
Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S Ryoo, Shrikant Kendre, Jieyu Zhang, Shaoyen Tseng, Gustavo A Lujan-Moreno, Matthew L Olson, Musashi Hinck, David Cobbley, Vasudev Lal, Can Qin, Shu Zhang, Chia-Chih Chen, Ning Yu, Juntao Tan, Tulika Manoj Awalgaonkar, Shelby Heinecke, Huan Wang, Yejin Choi, Ludwig Schmidt, Zeyuan Chen, Silvio Savarese, Juan Carlos Niebles, Caiming Xiong, and Ran Xu. xgen-mm (blip-3): A family of open large multimodal models, 2025. URLbttps: /ttpiiiiivv././////i//.aa.//..aa www. abs Ge Yan, Kris Wu, and Xiaolong Wang. uesd kitchens Dataset. August 2023.
15

---

Published as a conference paper at ICLR 2026
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. technical report. arXiv preprint arXiv:2412.15115. 2024.
Zhenjie Yang, Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuekai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-experts for vision-language-nction model in end-to-end au- tonomous driving. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of-thought reasoning for vision- language-action models. arXiv preprint arXiv: 2200.....00......22222222222..00 2025a.
Wei Zhao, Pengxiang Ding, Zhang Min, Zhefei Gong, Shuanghao Bai, Han Zhao, and Donglin Wang. Vlas: Vision-language-actinn model with speech instructions for customized robot manip- ulation. In The Thirteenth International Conference on Learning Representations, 2025b.
Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d vision lsson-languuag--actoon generative world model. In International Conference on Machine Learning, pp. 61229-61245, PMLR, 2024.
Allen Zren. open-pi-zero.
https: https://github... com/ allenzren/openet http:///rgiaa..cccaaaaaaee/////r pi zero 2025.
16

---

Published as a conference paper at ICLR 2026
Appendix
A
THE USE OF LARGE LANGUAGE MODELS (LLMs)
We affirm that this paper is prepared and written entirely by We did not use any large language models (LLMs) to generate the abstract, content, or any substantive part of the text. All ideas, analysis, and conclusions are the sole product of the authors' original thought and research. LLMs were employed solely for polishing grammar and refining phrasing, similar in scope to conventional grammar or style checkers.
B LIMITATIONS OF INTERLEAVE- VLA
While Interleave-Ve VLA exhibits strong generalization, training and deployment with interleaved puts increases computational cost due longer image-token sequences. Nevertheless, as discussed in Appendix [.3] this added cost is minimal in most practical scenarios. Regarding further improve- ments, future work could investigate more efficient image-token compression strategies and extend VLA models to support interleaved outputs-such as text or images-alongsil tion recent studies suggest may further improve performance (Intelligence al., [2025] ZZao al., 2025a). Moreover, from a data-pipeline perspective, although not readily observable in this paper, certain failure modes may potentially degrade Interleave-VLA performance in edge cases. Please refer to Figure 7 and Table for a detailed failure case analysis.
actions, a direc-
C INTERLEAVE-VEL TRULY UNDERSTANDS OR VISUALLY OVERFITS?
It is of value to know whether Interleave-VIl VLA truly grounds its decisions in visual inputs, or whether overfits to certain spurious correlations in the image or the prompt format itself. To this end, we design controlled probes that explicitly disentangle textual and visual information and analyze how Interlcave- VLA behaves under alignment and contradiction. It is important to understand whether Interleave- VLA truly grounds its decisions in visual inputs, instead overfits spurious corre- lations in the image or the prompt format itself. To this end, we design controlled probes that explicitly disentangle textual and visual information and analyze how Interleave-VLA behaves un- der both alignment and contradiction between the two modalities.
Understanding image-text contradiction. To study grounding under confticting modalities, consider a color-conditioned pick-and -nd-ce task: "Pick up [green/ycllow] block and place it on the towel." We compare five instruction formats (illustrated using the "green block" case):
1. text: "Pick up the green block...
interleave: "Pick up the [Image of green block] ..."
interleave-aligned: "Pick up the green block [Image of green block)...'
4. interleave-comtradict (image correct, text wrong): "Pick up the yellow block (Image green block]
5. interleave-contradict (image wrong, text correct): "Pick up the green block [Image of yellow block]
We report two metrics: (1) Success Rate, the fraction of episodes which the correct block is picked and placed; and (2) Intention Accuracy, the fraction of episodes in which the chosen block matches the task objective. Results are summarized in Table 7
From these comparisons, we observe that Interleave-VLA. is able to jointly interpret image and text modalities (first three columns). the last two columns, it is notable that Interleave-VLL. consistently prioritizes textual instructions over visual cues when the two conflict. This behavior clearly reflected in the interleave-contradit.. model achieves 100% intention accuracy, and in the contreare-cottrrdiic---/ setting (image correct, text wrong), where the intention accuracy drops to 0%. (The small 4.2% execution success rate in this setting stems from cases where the model initially places the wrong object the towel but
setting (image wrong, text correct), where the
17

---

Published as a conference paper at ICLR 2026
Metries / Tormat
Text
Interleave
Interleuve Aligged
(image cornect, lest wrong)
Interleaee-Comm Contrale-
limage wreng, sest correcti 62.5% 78.68%
Interleave- Contradict
Success (yellow
93.8% 97.9%
95.8% 7 79..%%
59.6/6 93.8%
4.2%
Accuracy green)
100.0%
100.0%
10000%
0.0%%
100L0%
Table 7: Grounding under image-text contradiction. Success Rate and Intention Accuracy (in %) of Interleave- -L on the color-conditionas block manipulation task under different instruction formats. First columns show that Interleave- VLA understands multimodality accurately and last columns imply that Interleave-VLLe consistently learns to attend to text when modalities contradict each other, which is evidently not driven by hallucination.
Metries/ Style
Normal
OCR
Quick
Abstract
Misleading
Ambiguons
Stnceess Rate tyellow green)
95.8% 8%, 789 6%
93 100.0% 100.0%
791.7% 7%
91.7%
38.35
56.3%
14.6%
37.5% 8.366
20.8
16.7%
Intention Accurncy (yellow/ green)
100.0% 100.0%
100.0%
91.7%
70.8% / 22088%
56.3% 8.3%%
35.4% 66.7%
Table 8: Performance across sketch styles. Success Rate and Intention Accuracy (in %) of Interleave- VLA when the target object is specified by sketches with different levels of clarity and ambiguity.
subsequently corrects its action.) Overall, these results indicate that the model genuinely under- stands the combined image-text prompt rather than merely overfitting specific visual patterns in the prompt image, as further supported by the new Tables 9 and 10 in the updated manuscript.
Understanding complex sketches. To further probe visual grounding beyond simple object photos, we revisit the task in QI and replace the reference image with human-creater sketches of varying informativeness and ambiguity. We categorize sketches into the following styles:
Normal: Detailed sketches drawn by humans in approximately 15 seconds.
2. OCR: Simple drawings of square/cube with the caption "yellow" or "green".
3. Quick: Rough sketches produced in under 5 seconds.
Abstract: Minimalistic sketches a square/cube labeled only with "G" or "Y" (for green/yellow).
5. Misleading: Intentionally confusing sketches, e.g., yellow-outlined cube with the caption "green".
Ambiguous: Under-specified sketches that omit clear color information or only loosely highlight the target object (e.g., a rough circle around region in the scene).
Success Rate and Intention Accuracy for each style. As sketches become less infor- mative and require more common sense reasoning interpret, performance degrades substantially.
These results show that Enterleave-VLA can reliably interpret rich, well-formed sketches (Normal, OCR, Quick), but its performance drops with highly abstract, misleading, or ambiguous sketches, where successful grounding requires additional high-level reasoning about the sketch creator's 's in- tent.
Conclusion. Taken together, the contradiction and sketch experiments provide strong evidence that Interleave-VI.A truly understands visual prompts, including shape and color (Normal), content (OCR sketches), and cross-modal consistency Pitterlaav-CContradee treating the image as a generic conditioning token tied to the accompanying text: in our experiments, Interleave- VLA successfully executes instructions when the target object is specified either in text or in the image, including cases where the two modalities disagree. At the same time, the degradation on abstract and ambiguous sketches highlights important avenue for future work: equipping interleaved lasion-language action models with stronger reasoning capabilities to robustly interpret under-speciffed systematically misleading visual prompts.
The model is not simply
D INTERLEAVE- VLA IMPLEMENTATION DETAILS
We extend two state-of-the-a VLA models, To (Black al. 2024 and Open (Km et al. 2024), to develop Interleave- VLL. While VLA models encompass a wide range of architectures (In-
telligence al. 2025 Team al. 2025 Bjorck al. 2025 Liu et al. 2024b: Shi et al.) 2024
18

---

Published as a conference paper at ICLR 2026
Normal
OCR
Quick
Abstract
Misleading
Ambiguous
Yellow
Ye (low Care
Yellow
Green
Crran Cioe
Green
Table 9: Example sketches for each style. Yellow-cube and green-cube sketches representing each drawing style. The examples are sourced from multiple individuals and are designed probe Interleave-VLL. VLA's ability to understand different aspects of a single object.
Detection Success (382)
Total (400)
Command Success (394)
Detection Failure (12)
Wrong Object ID (6)
No detection (4)
Command Failure (6)
Incomplete Bbox (2)
Wrong Command (3)
Unclear Command (2)
Incorrect Command Parsing (1)
Figure 7: Interleave-VLA. data-pipeline failure analysis. expand evaluation to 400 exam-
ples and quantify failures across the pipeline stages, from command parsing to visual grounding. The majority of samples (394/400) pass the nommand-parsing stage, while detection failures ac- count for most of the remaining errors (12/18). Each branch is annotated with the dominant failure modes, including wrong object identity, missed detections, incomplete bounding boxes, ambiguous commands, and incorrect command parsing. For example driven reviews, please refer to Table TO
Brohan et al. 2022 2023 Team et al. 2024| Chi et al. 2023). we focus on those based on VLM
backbones due to their inherent ability process image-text pairs. However, our approach is not restricted to VLM-bbssed methods and can be extended to other sequence modeling approaches for action prediction (Chi al., 2023 Team et al. 2024 Liu et al. [2024b] Brohan al., (022). The key modification involves interleaving image and text embeddings within the input sequence. Investi- gating the feasibility of this modification for other sequence modeling VLAs is exciting direction for future research. In this work, we focus and provide adaptations of Interleave- VLA from To and Open VLA the following sections in more detail.
Vanilla-VIa
AGrip
Interleave- VLA
aörp
Multimodal Large Language Model
Multimodal Large Language
Takenizer
Tikeeiow
Tokunbari
paher
Pick up the
place Into the
Figure 8: Comparison of Interleave- VLA and Extt architectures. Interleave- VLA is model- agnostic and requires minimal modifications to existing VLA architectures. The only change is the input format, which allows for interleaved image-text instructions.
19

---

Published as a conference paper at ICLR 2026
Failure Mode
Original instruction
Task image
Expected result
move light switch right
No Detection
take cucumber out cup
Wrong Object ID)
move the yellow block from the top the stack to the table
Dafreted
Incomplete BBox
Carrot is not present in the seene (only sushi). This comes from erross of orig- inal Open-X dataset.
place the carrot the middle
Wrong Command
The instruction is ambigu- ous: does not specify which object to put into the drawer. A disambiguated instruction (e.g., "put the red block in the drawer")) is expected. This comes from errors of original Open-X dataset.
put the object in the drawer
Unelear Command
The LIM parser fails treat "blue curl block" single object. Therefore, the object detection module is unable to retrieve the cor- rect blue block from other colors.
move the blue curl block from the cup to the top of the cube
Incorrect Command Parsing
Table 10:
Qualitative examples of primary failure modes in the dataset-constructiin pipeline. For each eтror type, we show the original natural-language instruction, the corresponding task image, and the analysis of expected detection outcome.
D.1 INTERLEAVE- VLA mo
We make minimal architectural changes to the (Black et al. 2024 model: only the input proces- sor. Specifically, to enable interleaved image-text instructions, we extend tokenizer vocabulary by introducing special tokens <BOI> (beginning of image) and <EOI> (end of image). These newly added tokens are used to delincate image embeddings within the instruction sequence. Specifically, the input tokens constructed follows:
<BOT> <i <image>1 ... <image>2s6
<text>
<image>2s7 ... <image>512
<BO1> <text> <HO1> <image>sıs <image>7e8 <HO1> <text>
Here, each <image> token represents a patch embedding from the visual encoder, and the <BOI> and <EOI> tokens mark the boundaries of each interleaved image segment. This design allows the model to flexibly process multimodal instructions by alternating between image and text tokens within unified sequence.
20

---

Published as a conference paper at ICLR 2026
Our Interleave- VLA approach is both effective and model-agnostic,, requiring only minimal modi- fications. Its effectiveness is evidenced by substantial improvements in generalization performance over To. achieving x gains as shown in Table 2 and 3 Interleave- VLA is model-agnostic,, seamlessly integrating into existing VLA models without requiring assumptions about the VLM. In Interleave-VL.. based To, the VLM backbone Paligemma (Beyer et al. 2024) demonstrates compatibility despite not being pre-trained on Internet-scale interleaved image-text data. More- over, our approach introduces only minimal modifications, with no architectural changes needed for the underlying VLM backbone. These facts highlight the practicality and broad applicability of Interleave- VLA for advancing multimodal robot learning.
D.2 INTERLEAVE-VLA OPENVLA
While architectural changes are not required to the VLM backbone as demonstrated in our adap- tation from TT- we further investigate whether modifying the backbone architecture affects its ef- fectiveness. Specifically, we replace OpenVLA' original Prismatic VLM (Karamcheti et al.] (024) backbone with VL2. (Chen et al. 2024b), which inherently supports interleaved image- text format. As shown in Figure 6 our Interleave-VEl VLA adaptation based Open VLA continues to function effectively, achieving more than double the performance of the original Open VLA. This result further highlights the model-agnostic nature of Interleave-VLL and its compatibility with di- verse VLA architectures. We have tested on different VL.M backbone for OpenVLA in Table and found that changing OpenVLA's VLM backbone has negligible effect performance.
D.3 INTERLEAVE- VLLA INFERENCE SPEED
To clarify the computational overhead introduced by Interleave- VLA during inference, we note that although the attention cost scales quadratically with the number of input images, the corresponding coefficient is small in practice. Consequently, the additional latency remains modest for typical real- world settings where the number of images is fewer than five. Empirical benchmarks conducted on an RTX 4090 GPU using our implementation of xy show that the inference latency is well approxi- mated by:
1.2n + 1.5n + 221,
indicating that the dominant term is effectively constant. When n < 5, the latency increase stays below 50 ms. Figure 9 reports the measured latency and percentage increase for different numbers of images, confirming that the overhead remains limited under common usage patterns.
In all tasks studied in this work, such latency differences do not affect control performance. For highly dexterous or fast dynamic settings where even small delays may influence throughput, the impact be mitigated via Real-Time Control (RTC) mechanisms demonstrated in prior work Black et al. (202).. Furthermore, the overhead can be reduced by employing high-speed im- age tokenizers (e.g., Vasu al. (2023)), as the image tokens are primarily used to encode semantic content rather than fine-grained spatial detail.
Method (VLM, Input)
L1
L2
L33
Interleave-Vil VLA (L.2, Interleaved)
83.14
58.14
64.00
OpenVLA (Prismatic, Text)
53.71
23.00
23.86
OpenVLA (1nterVVL..5,, Text)
45.29
24.71
29.71
Table 11: Comparing Interleave- VLA and OpenVLA with different VLM backbones on VIMA- Bench.
E RELIABILITY ANALYSIS OF THE INTERLEAVED DATASET GENERATION PIPELINE
Figure I0 illustrates the two complementary stages of our generation pipeline: Owlv2 and Qwen L+AAM. Empirical observations indicate that QwenVL+SAM excels at handling open- world objects, such the green star shown in the top right of the figure. However, it struggles
21

---

Published as a conference paper at ICLR 2026
0.36
Quadratic fit: 0.0012x² + 0.0015x + 0.2216
Inference Time (in seconds)
0.34
0.32
0.30
0.28
0.26
0.24
0.22
2
4
6
8
10
# #images
Figure 9: Interleave-VLA Inference time w.r.f number of images. When number images 2, it is typically the cost of TeXt-VLA model. Interleave-VL... takes in more images because of interleaving them instruction. While the inference cost scales quadratically with the number of input images, the coefficient very small compared the constant term. As is typical in most tasks (usually image number under 5), such modest latency increases do not incur performance.
in cluttered scenes under occlusions, depicted on the left side of the figure, where Owlv2 demonstrates superior performance. Notably, the combined approach significantly reduces failure rates, although both methods face challenges under severe occlusions low image resolution.
To evaluate accuracy, we randomly sampled 200 examples from the generated dataset and verified whether the detected images matched the corresponding text. Each sample may contain multiple key objects, and we considered it a failure if any key object was not detected. The individual error rates for Qwen VL+SAM and Owlv2 were 22. % and 17.4%, respectively, while the combined approach reduced the eтrоr rate to just 4.4%. These results highlight the effectiveness of integrating these two models enhance the reliability of the generation pipeline.
We then expanded our analysis to a larger set of 400 cases and categorized the dominant fail- modes each stage of the data-construction pipeline, ranging from LLM command parsing to object-detection grounding. As shown in Figure 7 the most frequent issues arise from visual grounding errors, such wrong object identification, missed detections, or incomplete bounding boxes, followed by command-level ambiguities and parsing mismatches. These systematic error patterns highlight (1) the existing robot dataset is not perfect in annotation and (2) both LLM pars- ing and visual detection modules could be further improved. For additional qualitative examples illustrating these failure categories, please refer to Table 10
F HALLUCINATION ANALYSIS OF VLA MODELS
While Figure provides a qualitative examination of VI.A hallucination patterns as revealed through attention visualizations, further conduct rigorous quantitative analysis of hallucinations in the CemplerEnv-Bridgee experiment. Existing callucination-evaluatiin methodologies designed for vision-language models (VLMs) do not directly transfer to vision- waaae-llanggggattteet (VLA) models, as VLAs do not generate textual outputs but instead execute action trajectories.
To address this gap, manually inspected and categorized hallucination failure modes across all rollouts. These failure modes are divided into high-level and low-level categories. High-level fail- ures include litter, where the VLA fails to infer the task intention and the robot arm jitters drifts,
22

---

Published as a conference paper at ICLR 2026
Target: Tray
CORRECT!
CORRECT!
NO DETECTION
WRONGE
Target: Green Star
NO DETECTION
NO DETECTION
Target: Paper Bag
Figure 10: Red: VLAAM and Yellow: Owlv2. Individual error rates 22.1% 1% and 17.4%, respectively. The combined error rate reduced to 4.4%.
and Wrong Object, where the model confidently selects an incorrect object. Low-level failures consist of Grasp Failed and Place Failed, corresponding to execution-level errors. Each rollout assigned single failure category, as the categories mutually exclusive.
We evaluate the two baselines introduced in Table To (Text- VLA) and Tо (Interleave- VLA full). The aggregated results presented in Figure and Table 12 The comparisons show that Interleave- VLA substantially reduces high-level hallucinations relative to Text-VL, shifting most of its errors from intention-level misunderstandings to lower-level execution failures.
Looking ahead, further progress hallucination mitigation for VLA systems will likely require automatic and scalable methods for quantifying hallucination rates in action trajectories.
Failure Mode
Model
In-domain (%)
Visual (%)
Novel Object (%)
Novel Category (%)
Interleave-VLA
0.0
0.0
0.0
3.4
Jitter
Text-VLA
0.0
0.0
6.5
27.5
Interleave- VLA
0.0
0.0
15.3
4.4
Wrong Object
Text-VLA
0.0
0.0
47.4
36.0
Interleave -LA
21.5
22.9
16.2
38.1
Grasp Failed
TLA
21.8
26.5
9.3
15.5
Interleave- VLA
7.3
3.6
14.6
0.0
Place Failed
Text-VLA
9.0
2.1
6.0
0.0
Interleave-VL.. VLL
28.8
26.6
46.1
45.8
Total
Text-VLA
30.8
28.6
69.2
79.0
Table 12: Detailed breakdown of hallucination failure modes for To (Text-VLA) and Interleave-ViL across all task categories. The table reports the per-scenario failure rates for each failure mode: high- level intention errors (Jitter, Wrong Object) and low-level execution emoos (Grasp Failed, Place Failed). Interleave VLA exhibits substantially lower high-level hallucinations, particularly in out- of-domain settings (Novel Object, Novel Category), while most remaining errors arise from low- level action execution. In contrast, Text-VLA shows markedly higher high-level intention failures, leading to increased overall hallucination rates. These numeric results correspond the aggregated visualization shown in Figure
23

---

Published as a conference paper at ICLR 2026
In-domain
Viuua
0.7
Text VLA
Text
0.4
011
jlter
Wring intento
Grap faiie
Race Faind
Total
Aiter
Wrog istonteon
Grapp Falred
Flace Faded
Tutal
Nevel Object
Novel Category
0.7
Text-/LA
Texi-LA
Ocommonce
Decumente
0.4
0.3
0.
00
Itoer
Weig interton
Grapp Rated
mace Patidd
Total
Atter
Wrong Intentonn
Gomp Fatted
pake Rublel
Tobal
Figure 11: Quantitative hallucination analysis of To with text-only instructions (Text-VLA) and interleaved image-text instructions (Interleave-VI..) VL.).. Across all task categories, Interleave- VLA achieves higher overall success rates. Each failed rollout is attributed to single failure mode: high- level intention errors (Jittr,, Wrong Intention) or low-level execution errors (Grasp Failed, Place Failed), Interleave VLA substantially reduces high-level hallucinations, with most residual failures arising from low-level action generation. In contrast, Text-VLA exhibits significantly more high- level intention errors, particularly in out-of-domain scenarios, leading to reduced overall success.
24

---

Published as a conference paper at ICLR 2026
G EVALUATION DETAILS
G.1 EVALUATION ON SIMPLERENV
G.1.1 SIMPLERENV EVALUATION TASKS
Our evaluation on SimplerEnv (Li et al. 2024) includes both In-Domain and Out-of-Domain tasks. The In-Domain tasks follow the original SimplerEnv WidowX BridgeData V2 Visual Matching setup. Since SimplerEnv tasks use text-based instructions, we adapt them into interleaved image- text instructions using the method described in Section 3.3 based on the first frame of the rollout before the robot arm begins moving.
In the WidowX BridgeData V2 setup, SimplerEnv does not support generalization tasks (referred to as the Variant Aggregation setup). To overcome this limitation, we introduce a set of challenging Out-of-Domain tasks inspired by the Open Vocabulary manipulation evaluations (Stone al., 2023). Unlike prior methods that rely on separate VI.Ms to detect target objects in the scene and inject this information into the robot policy, our Auerleave-VLA directly leverages interleaved image-text instruction to perform these tasks without requiring additional modules. These tasks are deliberately designed to be more challenging than the original SimplerEnv tasks, requiring the robot to generalize to novel objects and environments unseen during training on BridgeData V2 (Walke et al., 2023a).
We describe the 13 tasks (4 In-Domain and 9 Out-of- ofDDomain, as illustrated on the left of Figure 4 used in the SimplerEnv evaluation. The Out-of-Domail tasks are introduced in the order they appear from top left to bottom right, in Figure
1. widowx spoon on towel (In-Domain): This task is part of the original SimplerEny Visual Match- ing setting and is included in the BridgeData V2.
widowx carrot on plate In-Domain): Also from the original SimplerEnv Visual Matching set- ting, this scenario is present in the training data.
3. widowx stack cube (In-Domain): This stacking task is included in the original SimplerEnv Visual Matching setting and present in the training data.
4. widowx put eggplant in basket (In-Domain): This task is part of the original SimplerEny Visual Matching setting and is present in the training data.
widowx spoon on towel, unseen environment (Out-of-Domait Visual Generalization): The environment overlay is sourced from the RT-1 Dataset (Brohan al., (022) and is not seen during Bridge V2 training. The robot must generalize to a novel environment.
widowx spoon on towel, unseen tablecloth (Out-of-Domailo Visual Generalization): The table- cloth overlay is a random image from the internet, unseen in Bridge V2 training data, requiring the robot to generalize to new visual backgrounds.
7. widowx spoon on towel, unseen lighting (Out-of-Domain, Visual Generalization): The scene lighting changes dynamically with different colors (RGB) at 5Hz. The robot must generalize to novel and rapidly changing lighting conditions.
8. widowx redbull on plate (Out-of-Domain Semantic Generalization): This is an unseen object from a known category. While similar cans (e.g., tomato can) appear in training, the Redbull can is new. The robot must use language grounding to identify and manipulate the correct object among distractors (e.g., a Coca-Cola can).
widowx tennis ball in basket (Out-of-Domain Semantic Generalization): This is an unseen object from a known category. While similar balls (e.g.. white ball, blue ball) appear in training, the tennis ball is new. The robot must use language grounding to select and manipulate the correct object among distractors (an orange and ping pong ball).
10. widowx zucchini on plate (Out-of-Domain, Semantic Generalization): This task involves an unseen object from known category. While similar zucchini appears only once among 40,000 training episodes, this specific zucchini is entirely novel. The robot must leverage language grounding to accurately identify and manipulate the correct object, distinguishing it from dis- tractors such as a carrot.
11. widowx zucchini on saucer dish duces novel zucchini instance and an unfamiliar destination a saucer dish-bot of which
Semantic Generalization): This task intro-
25

---

Published as a conference paper at ICLR 2026
unseen objects from known categories. The robot must ground the instruction to correctly identify the target zucchini and place onto the saucer, discriminating it from distractors such as carrot and regular plate.
12. widowx toy dinosaur on towel unseen category. The robot must use language grounding to identify and manipulate the correct object among distractors (a toy elephant).
Semantic Generalization): This is a completely
13. widowx tape measure in basket (Out-of-Domail Semantic Generalization): This is a com- pletely unseen category. The robot must use language grounding to identify and manipulate the correct object among distractors (a purple eggplant).
14. widowx stapler on paper pile (Out-of-Domain, Semantic Generalization): This task involves a completely unseen category for both the object and the destination. The robot must leverage language grounding to accurately identify and manipulate the correct object (a stapler) among distractors (e.g., a spatula) and place it onto the unseen destination, the paper pile.
G.1.2 SIMPLERENV BASELINES
Our experiment in Table 2 compares Interleave-VLA (adapted from mo) with To (Black et al. 2024). RT-1-X (Brohan et al. 2022). and Octo-Base (Team et al.] 2024). RT-1-X and Octo models are evaluated using their official checkpoints and code, following the evaluation protocol in the Sim- plerEny (Li et al. 2024) repository. For тn. we use the reimplementation from the GitHub reposi- tory (Zren 2025). which is specifically trained on BridgeDaat V2 (Walke et al. (023a) and supports direct evaluation on SimplerEnv. Interlcave-VLA is built upon this reimplemented To codebase, with modifications to the input tokens and training on the interleaved BridgeData V2, using the in- terleaved dataset construction pipeline described in Section 3.3 To further highlight the benefits
of large-scale, diverse, crosseembodiment data, we also co-train Interleave-VLL with our curated
Open Interleaved X- Embodiment Dataset, as detailed in Section 3.3
Both Interleave-Ve VLA (including the co-trained variant) and mg models were trained with learning rate of 5e-5, global batch size of 1024, for approximately 30 epochs. The model input consists of a single observation image (no history), interleaved image-text instruction tokens, one proprioceptive token (no history), and four action tokens. Training takes roughly 2 days on 4XH100 GPUs with per device batch size of 16. Actions and proprioception across the diverse datasets are normalized to the 7D format: xyz position, Euler orientation, and gripper state, with all values scaled to the range [].,
The results presented in Table 2 reflect the best performance across checkpoints. Notably, perfor- mance can vary significantly between checkpoints, even among those that appear mostly converged. This variability is particularly pronounced for challenging tasks requiring precise manipulation, such as "widowx stack cube". These observations align with findings reported in the To reimplementation
GitHub repository (Zenn 2025).
G.1.3 SIMPLERENV EVALUATION RESULTS
Table provides detailed generalization results for the top-performing models: To. Interleave- VLA (adapted from тO), and Interleave-VLL co-trained, as reported in Table Interleave-VLA consis- tently surpasses To across all Out-of-Domain generalization tasks, demonstrating the effectiveness of multimodal learning from interleaved image-text data for both visual and semantic generaliza- tion. The co- co-trained Interleave-VLL model achieves further improvements, especially on semantic generalization tasks such as "RedBull on Plate," where similar RedBull cans are present in the RT-1 dataset for the Google robot. This highlights positive cross-embodiment task transfer to the Wid- owX robot. Overall, these results show that training with large-scale, diverse robot data enhances model generalization to novel tasks and robot embodiments, supporting approach of curating the Open Interleaved X- X-Embodiment Dataset.
Note that the Unseen Environment setting is omitted for the Interleave- LLA co-trained model be- cause the scene overlay is sourced from the RT-1 Google Robot dataset, which is included in the co-train data. As result, the model tends to generate actions intended for the Google Robot. Dur- ing evaluation, however, the robot used is WidowX, leading to a mismatch in embodiment and causing the model to produce incorrect actions.
26

---

Published as a conference paper at ICLR 2026
Table 13: Detailed evaluation results on Out-of-Domain generalization tasks based on SimplerEnv. Success rates (%) are reported for No, Interleave- VLA (adapted from To), and Interleave- co- trained with our Open Interleaved K-Embodiment Dataset, covering both visual and semantic gener- alization. Generalization results confirm that Interleave- VLA outperforms To across all tasks, with further mooosmembodimentt improvements from co-training.
Generalization
Semantic Genernlization
Model
Unseen Tabiecloth
Utscen Enviroement
Unseen Lighting
Redbull on Plate
Temnis Ball in Basket
Zurchini on Plate
Toy Dinosaur
on Towel
Tape Measure in Baoket
Stapler on Paper Pile
Average
78.0
77.0
59.2
0.0
30,0 73.0
50.0
24.0
1.0
38.0
39.7
Interleave VLA
G.2 EVALUATION ON VIMA -BENH
G.2.1 VIMA-BENCH EVALUATION TASKS
We evaluate performance on the majority of VIMA-Bench tasks, but excluding those requiring historical memory. Nemory-dependee
tasks are omitted because Interleave- LLA,, like common
VLA models Kim et al.] 2024 O'Neill et al.] 2024 Brohan et al.) 2022 2023 Black et al.) 22024] Team et al. 2025 Fang et al. 2025a Wen et al., 2025), is designed for meeooyyinddeendemn. first- order Markov settings. In general, common VLA models characterize the conditional distribution or), where A, = [a,, a.+1...., at-H-1] represents a sequence of future actions, and o, de- notes the current observation (comprising multiple RGB images, language command, and the robot's proprioceptive state). Extending VLAs to handle historical memory in interleaved instruc- tion scenarios remains interesting direction for future work.
VIMA-Bench employs interleaved image-text instructions for task specification. To evaluate text- instructed VLA models, we transform these interleaved instructions into text-only instructions by utilizing the shape and texture names provided in the VIMA-Bench codebase. For example:
Instruction: Put the into the
transformed Instruction: Put the rainbow triangle into the blue square.
G.2.2 VIMA-BENCH BASELINES
We evaluate Interleave- VLA (adapted from Open VLA) against several baselines: OpenVLA (Kim ctt al. 2024). VIMA-Gato (Jang et al. 2023 VIMA-Flamingo (Jiang et al. 2023). and VIMA- GPT Jiang et al. 2023). All models are trained on the same dataset generated using an oracle model, which has access to the exact 2D poses of all objects in the scene. This dataset generation process provided by VIMA. For Open the training data consists of text-instructed samples. Both intrlleae-VLLA and Open are trained on equivalent amount of the generated VIMA dataset using the following training hyperparameters: a constant learning rate of 2e-5 and global batch size of 128. This comparison demonstrates the effectiveness of Interleave- VLA in improving gen- eralization performance over existing VLA models. The results for VIMA-Gato, VIMA-Flame IIMA-Flamingo, and VIMA-GPT are taken from the original VIMA paper (Jiang al.) 2023) and serve additional benchmarks. These models, adapted by the VIMA team, serve as benchmarks to assess the progres- sion of VI.A models from earlier architectures like Gato, Flamingo, and GPT to the more advanced VLA
G.2.3 VIMA-BENCH EVALUATION RESULTS
The detailed results for the memory-indeppenden VIMA-Bench tasks presented in Table [4 The results demonstrate that Interleave-VLA benefits significantly from interleaved image-text instruc- tions, which enhance its ability to identify and manipulate the correct object by 2x. This approach proves more effective than relying solely on text descriptions to distinguish objects with the desired texture and shape among distractors.
27

---

Published as a conference paper at ICLR 2026
Table 14: Detailed VIMA-Bench results for LI, L2, and L3 level generalization evaluations. Interleave- VLA generally outperforms other VLA models and improves the generalization capacity of OpenVLA (Kim 2024) by over 2x.
VIMA-Bench L1
Model Name
task
task2
task3
task4
task
task
task15
AVG
Open VLA Interleave- VLL
VLA (Kim et al.
2024
70
78
92
49
53.71
87
82
81
54
82
100
96
83.14
VIMA-Gato
68
91
74
83
73.29
LIMA-Flamingo
56
58
63
48
62
66
40
56.14
VIMA-GPT
62
57
41
54
41
55.29
VIMA-Bench L2
Open VLA Kim et al.
2024
18
20
68
31
22
23.00
Interleave- VLA
36
32
75
26
100
94
58.14
VIMA-Gato
56.5
53.5
88
55.5
53
63
81.5
64.43
VIMA-Flamingo VIMA-GPT
51
52.5
61.5
49.5
55.5
82
42
56.29
52
52
49.5
54.5
51
76.5
43
54.07
VIMA-Bendd L3
Open VLA Kim et al. Interleave- VLA
2024
27
36
61
26
0
14
23.86
52
55
81
53
46
98
63
64.00
VIMA-Gato (Jiang et al. 2023)
58
84.5
56.5
49
52
59.43
VIMA-Flamingo (Jiang et al. 2023
49
50
66.5
47
50
66
30.5
51.29
VIMA-GPT (Jiang et al. 2023)
52
51
55
49.5
50.5
82
37
53.86
G.3
EVALUATION ON REAL ROBOT
G.3.1 REAL ROBOT EVALUATION TASKS
We evaluate two distinct manipulation tasks: Lift and Pick&Place, corresponding to the first and second rows of results shown in Table Visual illustrations of these tasks are shown on the right side of Figure The tasks are designed be challenging, requiring the robot to generalize to novel objects not seen during training. We describe these tasks in more detail.
The Lift task includes:
1. Lift pepper (In-Domain): 20 demonstrations collected with varied object arrangements and po- sitions.
2. Lift cup (In-Domain): 20 demonstrations collected with varied object arrangements and posi- tions.
Lift corn (In-Domain): 20 demonstrations collected with varied object arrangements and posi- tions.
4. Lift lemon (Out-of-Domain. Semantic Generalization): The target is an unseen object, as lemons included in the collected demonstrations, Although the lemon category appears in the pretraining data, it appears with different textures, robots, and environments. VLA models must utilize language grounding accurately identify and lift the target lemon among two distractor items.
5. Lift bean (Out-of-Domain Semantic Generalization): The target belongs to a completely unseen category, as beans absent from both the collected demonstrations and the pretraining dataset. VLA models must rely on language grounding to correctly identify and lift the target bean among two distractor items.
6. Lift spoon known category, as the demonstrations do not include this specific spoon. While the spoon cate- gory appears in the pretraining data, is represented with different textures, robots, and environ-
Semantic Generalization): The target is an unseen object from
28

---

Published as a conference paper at ICLR 2026
ments. VLA models must leverage language grounding to accurately identify and lift the target spoon among two distractor items.
The Pick&Place task includes:
Pick up kitchen cutter and place into the pot (In-Domain): 20 demonstrations collected with varied object arrangements and positions.
2. Pick up ladle and place into the pot (In-Domain): 20 demonstrations collected with varied object arrangements and positions.
3. Pick up pasta server and place into the pot (In-Domain): 20 demonstrations collected with varied object arrangements and positions.
4. Pick up the white and blue spatula and place it into the pot (Out-of-Domain, Semantic Gen- eralization): The target is an unseen object from a known category. The demonstrations do not include any spatula. While the spatula category appears in the pretraining data, it shown with different textures, robots, and environments. VLA models must utilize language grounding to accurately identify and manipulate the target spatula among two distractor kitchenware items,
5. Pick up the black and white spatula and place it into the pot -Out-of-Domain, Semantic Generalization): Similar to the previous task, but the target spatula is black and white. The robot must leverage language grounding to correctly identify and manipulate the target spatula among two distractor kitchenware items.
G.3.2 REAL ROBOT BASELINES
We compare Interleave- VLA (adapted from To) with pretraining against the following baselines: To with pretraining and Interleave- VLA without pretraining. The pretraining dataset is a subset of our curated Open Interleaved X-Embodiment Dataset, as described in Section [33] Interleave- VLA w/ PT is pretrained on this dataset and subsequently fine-tuned on the collected demonsirations from the FANUC robot arm before evaluation. For mo w/ PT, the same pretraining and fine-tuning protocol is applied, except the dataset is not interleaved. This setup allows for direct comparison to evaluate the benefits of interleaved image-text instructions for generalization. The Interleave- VLA w/o PT is trained exclusively the collected FANUC demonstrations, without exposure to the Open Interleaved X-Embodiment Dataset, enabling us to assess the impact of large-scale, diverse pretraining on performance. All models fine-tuned with learning of 5e-5, global batch size of 128, and evaluated across several checkpoints to mitigate the performance variability noted in Appendix G.1.2
G.3.3 REAL ROBOT EVALUATION RESULTS
Tables 15 and 16 present the detailed evaluation results for the Lift and Pick&Place tasks, respec- tively. Interleave- VLA, adapted from To. is compared against To and Interleave- VLA without pre- training (w/o PT). In generalization tasks, Interleave-Vi VLA consistently outperforms To in semantic generalization by 2x, highlighting the effectiveness of multimodal learning from interleaved image- text data. The results further demonstrate that pretraining Dataset significantly enhances performance across all tasks. For small-scale datasets (60 demonstra- tions in total per task), pretraining on the Open Interleaved X- X-Embodiment Dataset proves essential for achieving strong performance, as cross-embodiment pretraining enables the model to learn more robust representations and generalize effectively, even to the FANUC robot, which is not included in the pretraining data.
the Open Interleaved X-Fmbodiment
H SCALABILITY OF INTERLEAVE- VLA WITH THE OPEN INTERLEAVED X-EMBODIMENT DATASET
The Open Interleaved X- X-mbbddiment Dataset, detailed in Section 3.3 empowers Interleave- Veerreeve-VLAA.e to scale efficiently with increasing data. This section demonstrates the scalability of Interleave- VLA through pretraining and co-training strategies in varying data regimes.
Pretraining for Low-Data Regimes: As shown in Table 3, pretraining on the curated Open In- terleaved X-Fmbodiment Dataset is essential for achieving strong performance real robot tasks.
29

---

Published as a conference paper at ICLR 2026
Table 15: Detailed evaluation of the "Lift task". We conduct 12 trials for each object and report both the number of successful trials (#f Succ) and the number of trials where the correct object is manipulated Acc).
Interleave- VLA w/ PT Succ/ Acc
Interleave- VLA w/o PT # Suce Acc
my w/ PT Succ: / Acc
Category
Task
Trials
In- Do-Dommiin
pepper
12
7/12
2/4
7/10
In-Domain
corn
9/12
Q/4
4/12
n--Domain
cup
8/12
0/4
3/12
Out-of-Domand
spoon
12
9/11
0/2
9/11
Out-of- Domaann
hean
Out-of-Domanl
Jemon
8/12
0/4
2/5
Mean Success/ / Accuracy Rate
69.4 % 98.6 %
2.8 % 30.6 %
36.1 % 70.8 %
Table 16: Detailed evaluation on "Pick& task". We conduct 12 trials for each object and report both the number of successful trials (# Succ) and the number of trials where the correct object is manipulated (# Acc).
Interleave VLA w/ PT
Interleave- VLA wlo PT
To w/ PT Suce Acc
Calegory
Task
Trials
Acc
In-Domain
posta server
12
688
4/8
7/10
In-Domain
spoon
7/10
1/7
In-Domain
knife
12
4/7
2/7
4/12
Out-of-Domain
spatula
3/8
Out of Domaiin
black spatula
12
6/8
0/6
4/5
Mean Success / Accuracy Rate
43.3 96 /68.3
11.7 /
38.3 / /6.1.
This is particularly important due to the limited size of the FANUC dataset, which contains only 60 demonstrations per task. Pretraining on the significantly larger and more diverse Open Interleaved X-Embodiment Dataset enables Interleave- VLA to learn robust representations that generalize ef- fectively the FANUC robot, even though
included the pretraining data.
Co-Training for High-Data Regimes: Co-training with additional datasets from the Open In- terleaved X-Embodiment Dataset further enhances performance in semantic generalization tasks. While the Bridge Dataset V2 is already extensive and diverse, making substantial improvements challenging, co-training yields additional gains in semantic generalization. This demonstrates that interleaved training facilitates cross-embodiment skill transfer. Detailed results are presented in Table 17
Table 17: Scalability of Interleave- VLA through co-training on the Open Interleaved X-Embodiment Dataset, evaluated under the SimplerEnv Out-of-Domain setting. Incorporating datasets beyond Bridge Data V2 the Open Interleaved X- X-Embodiment Dataset further improves performance in semantic generalization tasks. The bold and underlined values represent the highest and second- highest scores, respectively.
Base Model
Paradigm
Co-trained
Visual
Novel Object
Novel Category
Avg.
3g
Black
2024
Interleave- VLA Interleave-VLA.
73.4
63.7
$3.0
63.4
71.5
70.7
57.3
66.5
I TASK FLEXIBILITY AND EMERGENT GENERALIZATION DETAILS
To highlight the task flexibility and emergent generalization capabilities of Interleave- VIAA when faced with unseen instructions, leverage the interleaved image-text interface to evaluate its per- formance across diverse user imput styles during deployment. The Interleave- VLA model used in this evaluation is directly taken from the SimplerEnv evaluation suite (Table 2 and Table without any additional fine-tuning. A summary of Interleave- VLA's performance statistics presented in Table
Below, we describe the three tasks and their corresponding prompts in the order they appear in Table
30

---

Published as a conference paper at ICLR 2026
1. Place (eggplant, carrot) the plate. Two types of instructions provided. The first row includes a hand-drawn sketch of an eggplant and carrot, created by a human on-the-fly -ly. The second row features sketch-style image of an eggplant and carrot sourced from the Internet.
2. Place {green, yellow) block on the towel. Two types of instructions are included. The first row contains a hand-drawn sketch green and yellow block, created by human on-the-fly. The second row features random images representing green and yellow block, sourced from the Internet.
Place (block, spoon] on the towel. Two types of instructions are used. The first row includes hand-drawn sketch of block and spoon, created by human on-the-fly. The second row features cropped images of the desired target objects, captured from a screen by a human on-the fly.
Interleave- VLA demonstrates remarkable emergent generalization capabilities, even when faced with diverse instruction styles such as Internet images, object crops (from a familiar input style but with unseen images), and sketches (a completely novel input style not encountered during train- ing). These emergent capabilities go beyond the typical generalization to novel objects and envi- ronments evaluated in prior VI.A models (Black et al. 2024, Kim et al., 2024). They highlight Interleave- VI.A's adaptability to new tasks and instruction formats, showcasing its practical flexibil- ity in processing diverse multimodal inputs.
J OPEN INTERLEAVED X X-EMBODIMENT DATASET DETAILS
The Open Interleaved X- X-Embodiment Dataset, curated described in Section 3.3 for training Interleave- -LLA, integrates data from sources within the Open X-Embodiment Dataset. To en- sure coherent training and facilitate cross-embodiment transfer, the action space across all datasets standardized to a unified 7D pose format: xyz position, Euler orientation, and gripper state. This normalization adheres practices established in recent VLA research (Kim et al. 2024 Black al.) 2024 Team et al. 2024). Our dataset features an extensive variety of over 3500 diverse object cat- egories, as depicted on the left of Figure 3 Additionally, Figure [2] highlights the wide range of skills encompassed within the dataset and provides a detailed breakdown of its composition and partitioning.
Interleaved X- X-Fmbodiment Dataset Composition
RT-1 (Brohan
2022
41.01
Bridge Waikee
2125a
28.25%
BC-Z [Jang ct al
2022
20.34%
Language Table (Lvnch al.
2023
7.81 %
UTAustin Mutex
2024
0.71%
Jaco Play Dass et al. 2023
0.51%
Berkeley Autolab UKS (Chen et al.
0.47%
IAMLab Pickup Insert (Saeeaa al. 2023
Stanford Hydra (Belkhale
2023
0.27%
U'Auusiin Sirius [L.ii et al.
2025h
0.26%%
UCSD Kitchen Yan ct al. 2074
0.07%
Figure 12: Left: Our Open Interleaved X-Embodiment Dataset is diverse in skills. Right: Compo-
sition of open data sources in our curated Open Interleaved X-Embodiment Dataset.
31


Reviews:
[{"review_id": "hnOgqbrgn6", "reviewer": "Reviewer_HzPe", "summary": "This paper introduces Interleave-VLA, a paradigm for robot manipulation that processes interleaved image-text instructions instead of relying on text-only instructions. The authors argue that text-only vla models struggle to generalize to unseen objects and scenarios, often suffering from attentional hallucinations where the model fails to correctly ground language to visual entities. To address this, they propose a model-agnostic framework that adapts existing vla models to handle mixed-media prompts. A core contribution is interleaved x-embodiment dataset of over 210k real-world trajectories, which was automatically generated by a pipeline that converts text instructions from the open x-embodiment dataset into an interleaved format. Through comprehensive evaluations in simulation and on real robots, Interleave-VLA is shown to offer two major benefits: 1) it improves ood generalization by over 2x compared to text-only baselines, and 2) it unlocks zero-shot capabilities, allowing the robot to follow instructions from novel visual modalities not seen during training, such as hand-drawn sketches, user-cropped images, and photos from the internet. The paper attributes this success to the explicit visual grounding provided by instruction images, which mitigates ambiguity and attentional failures.", "questions": "Please see weakness section. My main concern is 1) and 5). How do you disentangle the gains from the interleaving structure versus the benefit of providing any visual goal conditioning?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1) Introduced a highly effective paradigm for improving robot policy generalization by leveraging interleaved image-text instructions, achieving >2x performance gain on OOD tasks.\n2) A large-scale, real-world dataset (Open Interleaved X-Embodiment) and an automated pipeline for generating interleaved instructions.\n3) The work is validated through comprehensive experiments in both realistic simulation and physical settings.", "weaknesses": "1) The paper claims to be the first framework to comprehend interleaved image-text instructions and directly generate actions on robots in the physical world. however, interleaved multimodal prompts for manipulation were already explored in simulation [1](as in related work section) and explicitly framed as interleaving text and visual tokens. \n2) There seems to be no quantitative grounding/hallucination analysis, despite rich literature and metrics for VLM hallucination like POPE etc.\n3) FANUC evaluation averages over 12 trials per object and reports mean success/accuracy without uncertainty estimates; the paper does not report multi-seed variation for simulation either.\n4) Interleaving increases sequence length and thus quadratic attention cost; practical deployment on real robots is sensitive to inference latency.\n5) The authors attempt to isolate the source of performance gains by comparing Interleave-VLA (full) against a (partial) version trained on the same data but tested with text-only instructions. However, this analysis remains insufficient to attribute the gains specifically to the novel interleaving structure itself. The experiment conflates the effect of the input format (interleaving) with the effect of the input content (the addition of explicit visual goal images). The (partial) baseline lacks any visual goal information at test time, while the (full) version receives it so it is unclear how much of the significant performance jump is due to the proposed interleaving methodology versus the benefit of goal-image conditioning.\n\n[1] VIMA: Robot Manipulation with Multimodal Prompts. ICML 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZkkJGUTDxq", "reviewer": "Reviewer_AFur", "summary": "This paper proposes Interleave-VLA, a model-agnostic robot learning paradigm that enables vision-language-action (VLA) models to comprehend interleaved image-text instructions and generate continuous physical actions, alongside an automated pipeline to construct a large-scale real-world interleaved dataset from Open X-Embodiment. Comprehensive evaluations in simulation and real robots show Interleave-VLA doubles out-of-domain generalization to unseen objects compared to text-only VLA baselines and achieves zero-shot performance on diverse inputs like hand-drawn sketches and web images.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Interleave-VLA effectively addresses the critical limitation of text-only VLA models—attentional hallucinations (bias, diffusion, leakage)—by leveraging in-context visual grounding from interleaved instructions, with minimal architectural modifications to existing VLA models (e.g., π₀, OpenVLA) ensuring broad adaptability.\n\nThe proposed Open Interleaved X-Embodiment Dataset fills a key gap in robotic multimodal data, integrating 11 diverse real-world datasets with standardized actions and high-quality interleaved annotations, enabling scalable training and cross-embodiment transfer.", "weaknesses": "1. Computational efficiency is compromised: interleaved inputs introduce longer image token sequences, increasing training and inference resource demands, which the paper only mentions briefly without exploring mitigation strategies.\n\n2. To more comprehensively evaluate the effectiveness of the Interleave-VLA paradigm presented in this paper, it is suggested to introduce comparisons with additional recent VLA methods beyond the existing baselines to further validate its superiority in generalization and zero-shot capabilities.\n\n3. It is recommended that several VLA works in 2025 be incorporated and discussed to contextualize the novelty of the proposed approach, and clarify its positioning against state-of-the-art advancements in architecture design and data scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NiuqrDaFUZ", "reviewer": "Reviewer_tBwW", "summary": "This paper introduces Interleave-VLA, a new paradigm for robotic manipulation that enhances existing Vision-Language-Action (VLA) models by enabling them to understand and act on **interleaved image-text instructions**. The authors argue that traditional text-only instructions are often ambiguous and limit generalization, especially for unseen objects. To address this, their paradigm allows for more flexible and less-biased inputs, such as \"Put  on \".\n\nTo train this new paradigm, the authors developed a novel, automated pipeline to convert the large-scale Open X-Embodiment dataset into an \"Open Interleaved X-Embodiment Dataset,\" comprising 210k real-world episodes. The paper's core contributions are:\n1.  The Interleave-VLA paradigm itself, which is presented as a lightweight, model-agnostic adaptation for existing VLA models.\n2.  The large-scale interleaved dataset and the automated pipeline to create it.\n3.  A comprehensive set of experiments demonstrating that Interleave-VLA achieves 2x stronger out-of-domain (OOD) generalization in both simulation and the real world compared to text-only baselines.\n4.  A novel analysis of \"attentional hallucinations\" in text-only models, which Interleave-VLA mitigates.\n5.  Showcasing emergent, zero-shot capabilities, such as following instructions from hand-drawn sketches or web images.", "questions": "These questions are intended to clarify points that could strengthen my assessment.\n\n1.  **Probing the Grounding Mechanism:** Regarding the adaptation of $\\pi_0$, I am very impressed by the results given the base VLM was not pre-trained for interleaving. Have you run experiments to probe the nature of the learned grounding? For example, what happens in a contradictory scenario: if the text instruction is \"pick up the **red** block\" but the interleaved image prompt shows a **blue** block? Which instruction (text or image) does the model prioritize, and does this suggest a deeper \"comprehension\" or a learned attentional bias?\n2.  **Quantifying Inference Overhead:** Could you please quantify the computational overhead mentioned in your limitations? Specifically, what is the approximate increase in the number of tokens processed and the corresponding impact on inference latency (e.g., actions per second) when using a typical interleaved instruction (text + 2 images) versus a text-only instruction?\n3.  **Robustness of Sketch Generalization:** The zero-shot sketch generalization  is a fantastic result. How robust is this to the quality of the sketch? Could you provide a few examples of failure cases? I am curious about the limits of this capability, for instance, with sketches that are highly abstract, ambiguous, or could plausibly match multiple objects in the scene.\n4.  **Impact of Dataset Pipeline Noise:** Your dataset generation pipeline achieves a high 95.6% accuracy. For the 4.4% of failures, what is the primary error mode (e.g., wrong bounding box, wrong object ID)? Have you observed any impact, even minor, from this label noise on the final trained policy's behavior?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is a strong paper with several significant contributions.\n\n* **Solves an Important and Intuitively Clear Problem:** The paper's premise is compelling and well-motivated. Using a visual example (a crop, a photo, or even a sketch) is an immediately intuitive and powerful solution. The paper effectively translates this intuition into a practical robot learning paradigm.\n* **Comprehensive and Rigorous Experimentation:** The empirical validation is a key strength. The authors test their paradigm thoroughly across three distinct and challenging environments:\n    1.  **Simulation (SimplerEnv):** They demonstrate a 2x performance gain on semantically out-of-domain tasks over strong baselines like the base $\\pi_0$ model.\n    2.  **Real-Robot (FANUC arm):** They replicate these findings in the real world, showing a 2-3x improvement in OOD success rates when using their pre-trained model. This real-world validation is critical and very well-executed.\n    3.  **VIMA-Bench:** They show the generality of their *paradigm* by adapting it to a *different* VLA model (OpenVLA) and again showing superior performance, reinforcing their \"model-agnostic\" claim.\n* **High-Value Dataset Contribution:** The creation and open-sourcing of the \"Open Interleaved X-Embodiment Dataset\" (210k episodes) is a substantial contribution to the community. The automated pipeline (Fig. 3) built from modern foundation models (Qwen, OWLv2, SAM 2) to generate this dataset is itself a clever and valuable piece of engineering, achieving high accuracy (95.6%).\n* **Excellent Analysis and Insight:** The paper does not just present results; it provides a strong mechanistic explanation for *why* its approach works. The analysis of \"attentional hallucination\" (categorized as Attentional Bias, Diffused Attention, and Attention Leakage)  is insightful. Figure 5 provides a clear, qualitative \"Aha!\" moment for the reader, visually grounding the performance claims.\n* **Impressive Emergent Capabilities:** Perhaps the most exciting result is the zero-shot generalization to *unseen instruction modalities*. The ability to handle hand-drawn sketches , internet images , and user crops —without any fine-tuning on such data—dramatically expands the practical utility and flexibility of the system. This demonstrates a deeper level of generalization than just handling unseen objects.\n* **Clarity and Presentation:** The paper is exceptionally well-written and organized. The figures (especially 1, 3, and 5) are clear, informative, and effectively communicate the core concepts and results.", "weaknesses": "The paper is very strong, and the weaknesses are relatively minor or are areas for further strengthening.\n\n* **Quantification of \"Lightweight\" Adaptation:** The adaptation is described as \"minimal\" , primarily adding special tokens (`<BOI>`, `<EOL>`) to the tokenizer for the $\\pi_0$ model. While this simplicity is a strength, it's surprising that a model like Paligemma, which was *not* pre-trained on interleaved data, can learn this complex in-context visual grounding behavior so effectively from fine-tuning alone. The paper would be strengthened by a deeper analysis of this. Is the model truly \"reading\" the visual prompt, or is the prompt image just acting as a strong conditioning signal (e.g., a \"blue\" prior) that the model learns to associate with the text?\n* **Computational Cost Not Quantified:** The authors correctly identify a key limitation: longer image token sequences increase computational demands. This is a very practical concern for real-time robotics. However, this cost is not quantified. The review would be more complete if it included a brief analysis of the trade-off: What is the actual increase in inference latency (e.g., ms per action) when using an interleaved prompt (e.g., text + 2 images) compared to the text-only baseline?\n* **Analysis of Failure Cases:** The paper thoroughly documents the (many) successes of Interleave-VLA. However, a \"Failure Modes\" section is missing. When does Interleave-VLA fail? What if the user-provided sketch is highly ambiguous and matches *two* objects in the scene? What if the text (\"red\") and image (a blue object) are contradictory? Understanding the limitations of the proposed method is just as important as understanding its strengths.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1t7PXnNM85", "reviewer": "Reviewer_9vDg", "summary": "This paper presents Interleave-VLA that predicts robotic actions based on multimodal (vision and language) instructions. To train Interleave-VLA, the authors propose a framework that automatically converts the text instructions in the existing large-scale robot learning dataset (Open X-Embodiment) to multimodal instructions. Experiments were conducted on real-world and simulated environments, and results demonstrate that learning from multimodal instructions significantly improves out-of-domain generalization.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- [S1] The paper explores a novel direction where VLA models utilize multimodal language instructions, and the proposed approach (Interleave-VLA) is simple and does not require any architectural modifications.  \n- [S2] The experimental results seem interesting and clearly demonstrate the effectiveness of Interleave-VLA. \n- [S3] The paper is well-written and easy to understand. All figures and equations succinctly describe the details how Interleave-VLA works.", "weaknesses": "- [W1] When it comes to real-world scenarios, providing VLAs with multimodal instructions seems unnatural and cumbersome. Furthermore, additional complexity is required to convert textual instructions to multimodal instructions if users provide text (or verbal) instructions.\n- [W2] The performance of the proposed system is inherently dependent on the generalization capabilities of off-the-shelf detectors, such as OWLv2 and Segment Anything.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)