ChatVLA-2: Vision-Language-Action Model with Open-World Reasoning
Authors: Zhongyi Zhou
Yichen Zhu
Xiaoyu Liu
Zhibin Tang
Junjie Wen
Yaxin Peng
Chaomin Shen
Yi Xu
Venue: NeurIPS 2025 poster ()
Keywords: vision-language-action model
open-world reasoning
Primary area: applications

Abstract: Vision-language-action (VLA) models have emerged as the next generation of models in robotics. However, despite leveraging powerful pre-trained Vision-Language Models (VLMs), existing end-to-end VLA systems often lose key capabilities during fine-tuning as the model adapts to specific robotic tasks. We argue that a generalizable VLA model should retain and expand upon the VLM's core competencies: 1) **Open-world reasoning** - the VLA should inherit the knowledge from VLM, i.e., recognize anything that the VLM can recognize, capable of solving math problems, possessing visual-spatial intelligence, 2) **Reasoning following** – effectively translating the open-world reasoning into actionable steps for the robot. In this work, we introduce **ChatVLA-2**, a novel mixture-of-expert VLA model coupled with a specialized three-stage training pipeline designed to preserve the VLM’s original strengths while enabling actionable reasoning. To validate our approach, we design a math-matching task wherein a robot interprets math problems written on a whiteboard and picks corresponding number cards from a table to solve equations. Remarkably, our method exhibits exceptional mathematical reasoning and OCR capabilities, despite these abilities not being explicitly trained within the VLA. Furthermore, we demonstrate that the VLA possesses strong spatial reasoning skills, enabling it to interpret novel directional instructions involving previously unseen objects. Overall, our method showcases reasoning and comprehension abilities that significantly surpass state-of-the-art imitation learning methods such as OpenVLA, DexVLA, and $\pi_0$. This work represents a substantial advancement toward developing truly generalizable robotic foundation models endowed with robust reasoning capacities.

ChatVLA-2:
Vision-Language-Actton Model with Open- Embodied Reasoning from Pretrained Knowledge
Zhongyi Zhou Yichen Zhu 2-1
Xiaoyu Liu
Zhibin Tang² Junjie Wen
Yaxin Peng
Chaomin Shen
Yi Xu² Shanghai University
East China Normal University Midea Group
chatvla-2.githuu..o
Training
Modal
Robot Control In Word-World
SPuA sis sol the
Tole
Robot
Image-Text
Fareto Front
Model
Dyrank MAcE
VLMS
Charvind (oun)) aff
Jrthen
Aapott Durtor
VLAS
Robot Manipulation in Open-World
Mathematical Reasoning: Math Matching
Spatial Reasoning: Tay Placement
Hol
lotton
56-
4160A 1PApn)
in frovit tape
to the top Abotton the skej"
Act Siverigtt
The oae is en wooden aabi-.. i1 csn pioted 999,, 19.207
The orwn may is tocted juse ahel ahould placo
.R
Ro
56-40
This piet Oftere oft
Nehiae
Tep
Figure Our proposed ChatVLA-2 model enables generalized open-world reasoning and reasoning following abilities. We designed two tasks math matching game and toy placement experiment- demonstrate its generalization ability.
Abstract
lssionllnguagee-actin (VLA) models have emerged as the next generation of mod- els in robotics. However, despite leveraging powerful pre-trained Vision-Language Models (VLMs), existing end-to-end VLA systems often lose key capabilities during fine-tuning as the model adapts to specific robotic tasks. We argue that generalizable VLA model should retain and expand upon the VLM's core com- petencies: 1) Open-world embodied reasoning the VLA should inherit the knowledge from VL.M, i.e., recognize anything that the VLM can recognize, be capable of solving math problems, and possess visual-spatial intelligence, 2) Rea- soning following - effectively translating the open- reasoning into actionable
"Equal Contribution. Work done during Zhongyi Zhou's internship at Midea Group Corresponniig Authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

steps for the robot. In this work, we introduce ChatVLA-2, novel mixture-of- expert VLA model coupled with a specialized two-stage training pipeline designed to preserve the VLM's original strengths while enabling actionable reasoning. To validate our approach, we design a math-matching task wherein robot interprets math problems written on a whiteboard and picks corresponding number cards from a table to solve equations. Remarkably, our method exhibits exceptional math- ematical reasoning and OCR capabilities, despite these abilities not being explicitly trained within the VLA. Furthermore, we demonstrate that the VLA possesses strong spatial reasoning skills, enabling it interpret novel directional instructions involving previously unseen objects. Overall, our method showcases reasoning and comprehension abilities that significantly surpass ffute-oftthe-ae art imitation learning methods such as Open VIAA, Dex VLA, and Tn. This work represents a substantial advancement toward developing truly generalizable robotic foundation models endowed with robust reasoning capacities.
1 Introduction
I have seen further, is by standing on the shoulders of giants.
Isaac Newton
Vision-language-acee acctinn action models (VLAs) have become a popular approach for tasks in robotics ma- nipulation, navigation, and even full-body control. They have demonstrated remarkable capabilities in learning dexterous manipulation, tackling long-horizon tasks [ 2). and enabling open-world generation []] The success of VLAs, in contrast to traditional imitation learning methods, lies in their integration of pre-trained Vision-Language Models (VLMs). By leveraging the mature neural architectures from language models and multimodal networks, along with advanced training techniques and pre-trained knowledge from VLMs, VLAs significantly enhance robotic learning. This allows robots better understand and interact with the world while improving their ability to perform complex physical tasks.
Intuitively, pre-training VLA model consists of powerful, pre-trained VLMs, such as PaliGemma [5] or Qwen-VIL VL [6) should equip the robot with not only stronger vision-language fea- ture embeddings but also the comprehensive capabilities inherent VLMs including recognizing everyday objects, reasoning about spatial relationships, and solving mathematical problems. Consider simple task: writing down the answer to the equation 10 + 11 Such task is trivially easy for humans. A conventional hierarchical model would first leverage a pre-trained VLM to produce the answer (21), then invoke a low-level policy network to physically write it down. However, why might a VLA model struggle with such a simple task if it has never encountered the specific equation in its training data? In practice, tine-tuning on robotics-specific datasets often leads to the erosion of the original pre-trained knowledge from the VLM. For example, CL (7) illustrates that adapting a VLA model specifically for robotic control can cause previously acquired general knowledge to degrade significantly. As result, the VLA model may fail to accomplish tasks that seem trivial to humans, simply because these tasks were absent from the training dataset.
Such a gap leads to a natural question: How can we build VLA models that both keep their VLM prior intact and actively leverage it to achieve superior generalization in robotic control?
In this study, we introduce ChatVLA-2, a significant advancement toward achieving a truly general- izable robotic foundation model. The goal of ChatVLA-2 is not to construct an omnipotent robot model capable of executing every conceivable task. Instead, our primary objective is to demonstrate the feasibility of leveraging the pre-trained knowledge embedded within the VLM backbone. By doing so, we enable end-to-end robotic systems to generalize across diverse tasks, which traditionally require explicit planning by external agent. We argue that this generalization can be achieved by adhering to two fundamental principles:
Identifying overlapping feature spaces between multimodal understanding and robot control. Image-text data and robotic control data generally reside in distinct feature spaces, often resulting in competition for shared parameter spaces within models. ChatVLA ad- dresses this by employing separate static experts-onn one dedicated multimodal understand- ing and another specialized for robotic control- to ensure the clear separation of these tasks

---

into distinct feature spaces. This separation allows VLA models to excel independently in both domains. However, the isolated nature of these feature spaces currently limits the transfer of pre-trained knowledge to robotic control tasks. If mutual beneficial features could be effectively preserved and distinct task-specific features disentangled, the VIA model would be better positioned to intuitively leverage its pre-trained knowledge, thus significantly enhancing its generalization capability in robotic control.
Ensuring VLA models act according their intermal reasoning. Although VLA models demonstrate the capability for sophisticated internal reasoning, it remains uncertain whether their generated robotic actions accurately reflect this internal thought process. Previous research indicates that even large language models frequently produce outputs inconsistent with their thinking process 8 9. By ensuring that the action outputs through VLA models reliably follow their reasoning processes, we can substantially enhance their ability to generalize effectively across diverse and previously unseen tasks.
To achieve this, we propose a novel VLA model architecture employing dynamic mixture-of- experts within the VLM backbone. This design explicitly disentangles the feature spaces related to multimodal understanding and robotic action while adaptively identifying and preserving their shared representations. Additionally, we introduce a straightforward designed to align the action expert's output more closely with the model's internal reasoning process. Furthermore, we implement a two-stage training strategy: The initial stage aims to preserve pre- trained multimodal knowledge, simultaneously training robotic actions and establishing connections between these components. During the second stage, the VLM backbone is frozen, and only the action expert remains trainable, explicitly enabling it to learn to generate actions consistent with the internal reasoning derived from the upper levels of the model.
eessonnin-nnhannemmen module
To demonstrate the open-world reasoning and understanding capabilities of ChatVLA-2, we designed two tasks: a math matching game and a toy placement experiment. In the math matching game, we placed a whiteboard in front of the robot and wrote down a mathematical equation for the robot to solve. Several potential answers were placed before the robot, from which it had to select the correct solution and place it on the whiteboard. Importantly, we evaluated the robot entirely on -iittri-iitiittionn scenarios, meaning the presented equations never appeared in the training dataset. For evaluating spatial reasoning, we conducted toy placement experiment. In this task, the robot was instructed to pick up toy and place it at specific positions relative to various reference objects ge.g., to the right, left, front, behind, top, or bottom of objects). Many of the objects and directional instructions were entirely unseen during training. Therefore, this task required the model to accurately interpret the visual scene, reason about novel spatial instructions, and execute appropriate actions. Our experiments clearly illustrate the superior generalization capabilities of ChatVLA-2, particularly in reasoning and understanding tasks, surpassing existing mmitation-learning approaches such as Open VLA 10.. DLX 2 and mo [ This work represents a significant step toward the development of truly generalizable robotic foundation models that transcend the limitations of fine-tuning data by effectively leveraging pre- pre-trained VLM knowledge.
2 Related Work
Vision-language-action models in robot learning. Vision Sarton--nnnugg---teee action models (VLAs) form a
growing body of research within imitation learning 11 I3 24] that leverages pre-trained vision-languag
[4
15
16]
17
IS
19
20
21
22 23
models (VL.M)) as a backbone to enable both language comprehension and observational understanding. These methods typically fine-tune large pre-trained
VLMs to predict robot actions 25
26
27
28
29
30
31
32
33
34|
35
36
37
43 44 45
55 These methods have demonstrated strong performance across various simulated and real-world tasks, covering diverse robotic embodiments such as bimanual robots, mobile manipulators, legged robots, and humanoids. They also exhibit generalization capabilities across different environments and various objects. However, existing VLA models still lack the ability to generalize beyond the scope of their training data. Despite incorporating pretrained fail to effectively utilize the pretrained knowledge from these VLMs, limiting robots' capabilities for open- world manipulation. Consequently, this significantly undermines the rationale behind employing pretrained VLMs within large-scale models. In this paper, we introduce ChatVLA-22, a novel model designed specifically to retain and leverage pretrained VLM knowledge, thus enabling
49 50
51
52
53
39
54
models (VLMs) as their backbone, current VLA approaches
3

---

ChatVLA-2
Dynamic
Framework
Mixture of Experts
Language Instruction
Image Patches
Encoder
Router
K=2
LLM
N
Dynamic Mixture of Experts
N
Action Expert
On the table is 105/5..... Pick from inft.
MLP
Block 1
Block 16
Robot Action
Open-World Reasoning
Block 32
enhancement module is incorporated to ensure that the VLA model adheres to logical reasoning when performing actions. Right: Our method leverages a dynamic mixture-of-experts architecture to disentangle conflicting features between multimodal understanding and robotic control, while effectively integrating mutually beneficial features.
Figure 2: Model architecture. Left:
robots to perform open-world tasks effectively through pretrained reasoning and extensive general knowledge.
Embodied Reasoning in VLA models. A substantial amount of research has been dedicated to enhancing vision-language sation-lnguuge-acttion (VLA) models by incorporating the chain-of-thought (CoT) (56) methodology, inspired by the recent successes of large language models (LLMs) in various cognitive and reasoning tasks. The primary motivation behind adopting CoT is replicate the sophisticated reasoning and decision-making capabilities of LLMs within robotic systems, enabling robots to perform more complex, context-aware actions in dynamic, real-world environments. For instance, Embodied-CoTT (57) and CoA-VLA (58) utilize structured textual instructions enriched with spatial localization information, COT-VLA /VPP 660 integrates reasoning via generated visual imagery, DiffusionVLL (61. DexVLA (2]. and T0.5 3 rely on plain language instructions. However, in these models, reasoning whether represented through textual instructions or visual cues cues-is is explicitly trained and consequently limited to knowledge contained within the training datasets, restricting their capacity for broader generalization. In this work, we significantly advance this line of research by leveraging pretrained knowledge from VLMs, thereby empowering VLA models with enhanced open-world reasoning and generalization capabilities.
3 Methodology
This section introduces proposed ChatVLA-2 and is organized into three parts. Section 3. provides preliminary background on vision-languagc-atiion (VLA) models. Scction 3.2 details the neural architecture, and Section 33 presents the two-stage training strategy. Together, these components empower the VLA model with open-world reasoning and understanding capabilities.
3.1 Preliminary: Vision-Langungee-Action Model
VLA models, leveraging pre-trained VLM perception, are becoming a dominant approach in robotic control. Benefiting from large-scale multi-modal pre-training. VLAs demonstrate significant advan- tages in bimanual manipulation [1] long-horizon task planning [I 61], and mobile manipulation [3]. We adopt Dex VLA 2 as our foundational model architecture. Specifically, we employ the Qwen2- model as its core VLM. The image encoders project the robot's visual observations into the same embedding space as the language tokens. When handling multiple camera views, the visual embeddings from each view concatenated. The VLM component produces two types of
VL 62 6

---

outputs: reasoning tokens and action tokens. The action tokens undergo further processing through projection module composed of two linear layers and a LayerNorm layer. Additionally, we employ the pre-trained IB ScaleDPP63 63] module as our action expert. We chose Dex VIA because it is among the few open-source VIAA models that output unstructured textual reasoning, allowing our approach to effectively harness the VLM's pre-trained knowledge and enabling the VLA model to generalize across diverse scenes.
3.2 Model Architecture
Dynamic mixture-of-expert. Typically, VLA models utilize a dense vision-language backbone as their foundational architecture. Prior research 7 indicates that multi-modal understanding and robotic manipulation tasks often compete within the parameter space, causing dense VLA models to exhibit erosion of multi-modal comprehension capabilities. To this end, integrate a Dynamic Mixture-of Cftturreof-Expertt. (MoE) [64] architecture to effectively handle diverse and complex multi-modal inputs encountered in different tasks. Specifically, our approach utilizes adaptive routing strategy where expert modules are dynamically selected based on the characteristics of the visual and textual inputs. Idcally, we anticipate that some experts will specialize in task-specific features, such as multi- modal understanding and robot control. These experts focus exclusively on particular tasks, enabling them learn specialized feature representations through dedicated sets of weights. Conversely, other experts may capture mutually beneficial features shared across multiple tasks, such as spatial reasoning, which is critical for both scene understanding and manipulation. We also expect the gating network to utilize learned criteria to intelligently evaluate input data, selecting the most appropriate subset of experts for activation. This adaptive strategy ensures efficient allocation of computational resources and reduces unnecessary computations. We use the pre-trained MLP weights to initialize the MLP layers for the experts.
Why static/shared experts are not used? The key to enabling VLA models to generalize in open- world robotic manipulation lies in preserving the pre-trained knowledge. For architectures like Qwen2-V VL whose LLM component lacks native MoE support - introducing static or shared experts would disrupt the original model structure. Such architectural alterations risk rapidly degrading the VLM's pre-trained knowledge, compromising its reasoning capabilities. Dynamic MoE circumvents this issue by preserving the LLM's intact architecture while selectively activating expert modules. This approach ensures the foundational knowledge remains undisturbed while enabling task-specific adaptation. Our empirical studies in TableJconfirm that dynamic MoE critical for maintaining the open-world reasoning necessary for generalizable manipulation, as it balances knowledge retention with adaptive learning. In practice, we utilize total of eight experts and dynamically select two experts during inference.
Reasoning following enhancement module. A distinctive feature of our method is that the model not only follows given instructions but also aligns robotic actions closely with the generated reasoning. Prior approaches, such as Diffusion VLA 66 and VLA 2], utilize FiLM layers incorpo- rate reasoning tokens. These methods primarily handle in-domain reasoning scenarios typically encountered during training, making FiLM layers sufficient for reasoning alignment. In contrast, our approach deals with diverse, novel reasoning types not encountered in the training data. Therefore, our method requires a more robust and flexible VLA model capable of effectively following complex, out-of-distributio reasoning.
We introduce an enhanced reasoning-followw action models. Specifically, we replace the original observation embedding with reasoning tokens projected through MLP. This reasoning representation is then combined with the current timestep embeddings and used to condition the generation of scale and shift parameters, effectively injecting reasoning context into the model. Importantly, we incorporate this mechanism exclusively into the latter half layers, rather than uniformly across all layers. This design choice aligns with findings from prior studies, such as PointVLA deeper layers of action experts have a smaller impact on robot control. Our results demonstrate that this selective integration allows the model robustly handle open-world reasoning scenarios without sacrificing in-domain accuracy.
module designed improve reasoning capabilities in
(65 and GROOT N1 [66), which suggest that modifications to the
5

---

Robot Data
VLM
Action Expert
Action
Stage 1
VIT
Dynamic MoE
15
Co-training
Image-Text Data
10 5 =
In-Domain
Open-Worid X
Open-World Reasoning
VLM
Stage 2 Reasoning Following Training
Action
Action
Robot
Expert
VIT
6
Dynamic MoE
3
In-Domain
Open-Worid
Open-World Reasoning
Figure Training Strategy. We leverage two-stage training strategy. In first stage, we perform co-training on image-text data and robot data to empower VLA with open-world reasoning capabilities. In the second stage, we freeze the entire VLM and train only the action expert. thereby preserving open-world reasoning while enhancing cnstruction-oolowwiin abilities in VLA.
3.3 Training Strategy
Our previous section introduced the neural architecture of ChatVLA-2, which primarily focuses on enabling the VLA model to more effectively extract common knowledge from pre-trained data and robot actions, guiding the robot adhere argue that this alone insufficient for effectively training a general-purpose VLA model. Specifically, mixing image-text data and robot data during training makes challenging control the learning process effectively. To address this, we propose a dual-stage training strategy designed to enhance the smoothness of robotic control and increase the success rate of task completion.
closely the generated reasoning. However,
Empowering VLA with worn-world embodied reasoning and understanding. Co-training on image-tex and robot data is essential for enabling the robot foundation model to reason and understand scenes in the wild. During this stage, we train the model on both tasks, specifically using datasets COCO (67). TextVQA (68). and GQA [69]. We also construct dataset of image-text pairs involving robotics scenarios for fine-tuning purposes. Additional details are provided in the Appendix. We apply text augmentation techniques to increase query diversity across all training data. We deliberately avoid selecting training data to bias the VLA toward specific skills such as OCR, mathematical reasoning, or spatial reasoning, our goal utilize pre-trained knowledge for open- world manipulation.
For robot data, we collect 600 trajectories from a matching game and 300 trajectories from a toy placement experiment. Similar to Dex VLA and Wo.5, all robot data annotated with reasoning phrases. We maintain an image-text text data to robot data ratio of 1:3. This setup follows previous methods. The model undergoes training for 50k steps, beginning with initial learning rate of 2e-5 and a warm-up phase for the first 3k steps. Subsequently, we apply a cosine learning rate scheduler, scaling down the learning rate 2e-6.
Enhancing maasoning-folloingg in VLA. By jointly training the model on both image-text data and robot data, it learns to reason, recognize, and effectively act within open- world scenarios. The initial stage preserves a significant portion of the pretrained knowledge. However, since our method aims for robots to perform tasks in open-world environments, the reasoning required may not be presented in the training data. Thus, it becomes particularly crucial to strengthen the connection between reasoning and action, ensuring that actions accurately follow and execute the reasoning outcomes for generalizable robot control.
Specifically, we freeze the pretrained VLM and only train the action expert. By keeping the VL.M fixed, we effectively preserve pretrained knowledge acquired in the initial training stage. Consequently, the robot's actions guided not just by the initial language instructions and image observations but

---

Franka Setup
ARX Setup
Math Matching Game
Instruction:
Answer the question and pick the card with correct onswer.
Step
Steps
28+1
the board 18-16-34... Pick 34 from right
On board is 4+8, 4+8-12 There are 1[left), 2(middle), 7(right) First, pick from left
Then from middle
Camera
Toy Placement
Instruction:
(obi) place (olace] (torget)
Pick the avocado
Pick the corn
Pick the mug
Behind the
Left of pink block
Right of orange
Front of bowl
Top/Bottom of shelf
Figure 4: Experimental setup for math matching game and toy placement. We Franka Emika robot equipped with a Robotiq gripper to pick and place items at specified target locations. We utilize the ARX R5 bimanual robots with a top camera RealSense L515, Our experiments demonstrate that the proposed method successfully completes tasks involving previously unseen spatial instructions and novel objects.
also significantly by the reasoning outputs generated by the upper layers of the model. We found this strategy particularly beneficial in enhancing the model's 's understanding and responsiveness to previously unseen reasoning scenarios.
4 Experiments
In this section, we conduct extensive real-robot experiments to demonstrate that the end-to-end model capable of wooldworld reasoning and understanding and can effectively transfer this knowledge to interactions with the physical world. We do not evaluate using simulation benchmarks, as the VLA capabilities demonstrated by approach exceed what current simulation benchmarks assess.
We specifically select two scenarios math matching games and toy placement task compre- hensively evaluate our proposed method. These experiments examine the model's proficiency in mathematical reasoning, spatial reasoning. optical character recognition (OCR), and object recogni- tion and localization, most within an open-world context involving scenarios that were not part of the training dataset.
4.1 Mathematical Reasoning: Math Matching Game
Evaluation metrics. We report three types of metrics to evaluate the ability of ChatVLA-2 in manipulation, reasoning, and understanding in both in-domain and open-world. Manipulation success rate: We report the average success rate to measure whether the model completes the task or not. 2) OCR: For OCR, we assign point for correctly recognizing hand-written numbers, 1 point for identifying card values and their positions and points for correctly recognizing the sign. 3) Mathematical reasoning: For mathematical reasoning, we assign point for correct answer and point for correctly selecting the card.
Experimental setup. We consider both in-domain and worn-world settings. Specifically, for the in- domain evaluation, all numbers and mathematical symbols exactly match those in the training dataset. However, since numbers and symbols handwritten, variations in calligraphic style inevitably occur. For the open- apecbvoocbylary setting, the mathematical equations tested are entirely absent from the training data.
Robot setup. We utilize the bimanual, ALOOA-style robot arm system, ARX-R5, featuring two arms, each with 6 degrees of freedom (6-DoF) and equipped with top RealSense L515 camera.
7

---

This configuration results in a 14-dimensional combined state and action space. Data collection is performed through teleoperation equipment frequency of 50 Hz.
Experimental results. The experimental results are presented in Table 1 We compare our method against several tatt--of-the-art models, including Octo [70), Diffusion Policy 32 OpenVL. (O), GROOT NI [66], VLA [2]. ChatVLA 77]. and To []]. We first examine the in-domain performance. For mathematical reasoning and OCR tasks, only a few models such as DexVLA and ChatVLA can output language-based responses. They demonstrate reasonable accuracy in reasoning and OCR tasks, achieving performance comparable to ChatVLA-2. Similarly, in manipulation tasks, ChatVLA-2 does not significantly outperform models like To and DexVLA, which already exhibit near-perfect performance.
However, substantial differences emerge in open-world scenarios. Even ChatVLA despite its multimodal understanding capability, fails these tasks when the robot control expert is activated. Consequently, none of the compared methods successfully completed any manipulation tasks in open- woelworldd conditions. In contrast, ChatVLA-2 achieves meaningful performance: 3.58 in OCR accuracy, 1.73 in mathematical reasoning accuracy, and 82.7% manipulation success rate. These experiments highlight the core contribution of our approach: although it may not significantly outperform others in well-trained (in-domain) manipulation tasks, ChatVLA-2 demonstrates substan- tial superiority in open-world scenarios, successfully handling novel mathematical equations and unfamiliar typography. This represents significant advancement from zero to effective generalization capability.
4.2 Spatial Reasoning: Toy Placement
Evaluation metrics. We measure the model with three metrics. First of all, similar the previous experiment, we report the average success rate of robot action success. Additionally, we provide the open-world object recognition performance in the reasoning process. In the output reasoning, the model needs to output the bounding boxes for the objects that are targeted.
Experimental setup. We consider both in-domain and open-world settings. For in-domain evaluation, all objects appear in the training set. For open-world evaluation, the target and reference objects entirely unseen during training. The model must recognize all objects an open-world setting, identify the reference objects mentioned in the instruction, understand spatial relations, and execute the placement accordingly.
Robot setup. We utilize a 7-Degrce-of-Freede Freedom Franka Emika robot equipped with a Robotiq gripper. We use one ZED 2 camera positioned the right side. Data collection is performed using teleoperation equipment at a frequency of 15 Hz.
Experimental results. The experimental results are presented in Table 2 In the in-domain setting, our proposed method performing comparably VLA and ag- While ChatVLA was capable of recognizing novel objects in the open-world setting, its performance remained much lower than our method's 0.94. For action execution, models other than our method and To exhibited near-random success rates in this setting. Even ChatVLA, despite demonstrating some reasoning ability, showed limited open-world robot manipulation ability. In contrast, our method achieved an average success rate of 81.4%, representing 3.52-times improvement over VLA. This result highlights strong spatial reasoning capabilities and masoning-foolowing capabilities of our method in open-world scenarios.
4.3 Ablation Study
How important is mixture-of-expert in VLA? This section investigates whether the mixture-of- experts (MoE) mechanism in VLA is crucial for enabling VLA models to generalize for reasoning and understanding in an open- world setting. Specifically, using the exact same training configuration, we compare the baseline models that do not incorporate MoE. Since MoE introduces additional com- putational overhead during inference, further compare the model with larger VLA configuration, specifically the 7B VLM, which has significantly higher number of parameters at test time.
The experimental results are presented in Table We conducted experiments on the math matching game and observed a significant drop in the average success rate. We hypothesize that this decline due to conflicts in the parameter space between robotic actions and reasoning/understandiin
8

---

Table 1: Results on the math matching game. We evaluate multiple models on both in-domain settings, where the data is presented in the training data, and open-world setups. We evaluate average score of OCR (4 scores in total) and mathematical reasoning (2 scores in total), and average success rate of task execution both setups.
In Domain
Open-World World
Method
Reasoning Score
Success Rate
OCR Score
Math Reasoning Score
Success Rate
Octo 70
2/13
0/52
Diffusion Policy
7/13
3/52
OLe
2/13
0/52
GROOT N1
66
4/13
3/52
VeX
5.2/6
12/13
0.21/4
0.06/2
10/52
ChatVLA
5.8/6
10/13
1.08/4
0,42/2
4/52
1
12/13
8/52
ChatVLA-2 (Ours)
6.0/6
11/13
3.58/4
1.73/2
43/52
Table 2: Results the toy placement task. We evaluate multiple models on both in-domain settings, where the data is presented the training data, and open- world setups. We evaluate average object recognition score, spatial affordance score and task success rate at both setups.
Method Munipulation
In Domain
Open World Spatial Affordance
Object recognition
Spatial Affortance
Arg. Success Rate
Object recognition
Avg. Success Rate
Octo
Diffusion Palicy
$2/67
17/155
OL
10
23467
10/156
GROOT N1
12/156
2
ChaNVLA
0.97
60/67
0.71
0.35
22/156
(
61/67
/
25/156
ChatVLA-2 (Ours)
0.99
61/67
0.94
0.88
127/156
The mixture-of-experts approach effectively disentangles the feature spaces associated with these conflicting features. Furthermore, we find that increasing the number of parameters to 7B does not alleviate these conflicts. Upon investigating the cause of the failure, we discovered that for unseen mathematical equations, both dense models fail completely. By examining the mathematical reasoning and OCR scores, we find that when the dense models encounter unseen equations, they often fail to arrive at the correct answer and, in most cases, recognize the wrong answer instead.
Ablation study on two-stage training. Our paper proposes a two-stage training strategy designed explicitly to enable VIA models to act effectively open-world scenarios and consistently follow generated reasoning. Table 4 presents the ablation study isolating the effects of Stage 1 and Stage on model performance the math matching game. When Stage was excluded, the model's robotic control performance in open-world scenarios dropped to 23% under the same number of training steps. This suggests that while open-world reasoning is generated in Stage I, it has not been effectively injected in action execution. In contrast, removing Stage I resulted in near-zero score of open-world reasoning capabilities, including both OCR and mathematical tasks, which highlights the eritical role of co-training with image-text data.
4.4 Results on Multimodal Understanding and Visual-Question Answering
We have conducted extensive evaluations across 12 diverse multi-mode understanding benchmarks, covering tasks such as document understanding (DocVQA), chart and scientific reasoning (ChartQA, AI2D), DCR-basee question answering (TextVQA, OCRBench), and real-world fine- fine-grained recogni- tion (Info WerlQQA, MMStar). We also present the results of baseline model ChatVLAA, as is shown in Table 5
Table 3: Ablation on mixture-of-expert.
Table 4: Ablation on training strategy.
Method
OCR
Math
Avg.
Math Matching Game
Stage
Stage
Dynamic MoE
3.58
1.73
43/52
OCR
Math
Avg.
Static MoE + Dynamic MoE
2.38/4
0,92/2
11/52
3.20
1.33
12/52
Shared MoF Dynamic MoF
3.07/4
1.12/2
25/52
v
0.15
0.04
3/52
3B Dense Model
0.04
0.00
2/52
Dense Model
0.08
0.00
8/52
/
3.58
1.73
43/52

---

Table 5: Understanding task: Evaluation of VLAs 7 VQA benchmarks and 5 Multimodal Understanding benchmarks. Boldface denotes top-ranked methods.
Metiod
Parus
TextVQA
DreVQA
IntoVQA
VQQ Berehuaskis
AED
ChunQA
MIVQA
RealWeraRUA
MMIMU
Multinachi Underitaaling Bershmaric
MMStrr
MME
OCRllench
Hallllench
OgenVLAA
78
0
0
0
0
0
17.2
The results demonstrate consistent improvements 11 out of 12 benchmarks, with particularly notable gains in tasks requiring strong multi-modal understanding capabilities- such as +7.0 on TextVQA, +11+ +11.7 on ChartQA, and +6.4 on Info VQA. These results indicate that key understanding abilitics, including fine-grained recognition, OCR, and multimodal reasoning, are retained from the pre-trained VLM.
5 Conclusion
Imitation learning typically requires extensive data to master specialized skills for particular tasks. Developing models capable of reasoning and general understanding within open-world scenarios remains frontier research topic that has yet to be thoroughly explored. this work, we introduce ChatVLA-2, which endows vision- laaion-lnnugg-acttionn (VLA) models with the capability to perform diverse tasks by leveraging innate reasoning and understanding abilities derived from pretrained vision-language models in an end-to-end manner. Our core contribution is the introduction of dynamic Mixture-of -ffture-ff-Expetts (MoE) module integrated atop a pretrained vision- langun- backbone. This module efficiently manages different task requirements, where certain experts share common multimodal features, while others are dedicated to task-specifis representations. Additionally, we propose two-stage training strategy: initially, tions between pretrained multimodal knowledge and robotic actions; subsequently, we introduce censsoninn-follwwiinn stage, enabling the model to comprehend reasoning outputs and effectively translate them into corresponding actions.
guide the VLA model to establish connec-
Acknowledgments
This work is supported by the Sci-Tech Innovation Initiative by the Science and Technology Com- mission of Shanghai Municipality (24ZR 1419000, and the National Science Foundation China (12471501).
Author Contributions
Project Leader & Advisor: Yichen Zhu Paper Writing: Yichen Zhu, Zhongyi Zhou
Algorithm Development:
Zhongyi Zhou, Yichen Zhu, Junjie Wen Policies Training & Evaluation: Zhongyi Zhou, Yichen Zhu Data Processing: Zhongyi Zhou, Xiaoyu Liu
Leadership: Yichen Zhu, Chaomin Shen
References
[1] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman. Brian Ichter, et al. pi 0: A vasion-language-action flow model for general robot control. arXiv preprint 2.1.:24110.2464,, 2024.
[2] Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. Dexvla: Vision- language model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.05855, 2025.
[3] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. pi0.5: vision- laiion-anguage-ace model with open-world generalization. arXiv preprint ar0iv:2504 16054, 2025.
10

---

[4] Minjie Zhu. Yichen Zhu, Jinming Li. Zhongyi Zhou, Junjie Wen, Xiaoyu Liu, Chaomin Shen, Yaxin Peng. and Feifei Feng. Objectvla: End-to-end open-world object manipulation without demonstration. arxiv preprint arXiv:2502. 19250, 2025.
[5] Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang. Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin. Michael Tschannen, Emanuele Bugliarello, et al. Paligemma:
A versatile 3b vlm for transfer. arXiv preprint wxXiv2400.007220 2024.
[6] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, arXiv preprint urXiv:2308. 12966, 1(2):3, 2023.
[7] Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng. Ran Cheng, Yaxin Peng, Chaomin Shen, et al. Chatvla: Unified multimodal understanding and robot control with vision- haan--llangggaaeiiinn model. arXiv preprint arXiv:2502. 14420. 2025.
[8] Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don't always say what they think: Unfaithful explanations in thain-of-thought prompting. Advances in Neural Information Processing Systems, 36:74952-74965 2023.
[9] Wenjie Ma, Jingxuan He, Charlic Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.098555 2025.
[10] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-ctton model.
[11] Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple representations. In ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024.
[12] Yanjie Ze, Nicklas Hansen, Yinbo Chen, Mohit Jain, and Xiaolong Wang. Visual reinforcement lcarning with self-supervised 3d representations. IEEE Robotics and Automation Letters, 865)2290-28977, 2023.
[13] Kun Wu, Yichen Zhu, Jinming Li, Junjie Wen, Ning Liu, Zhiyuan Xu, Qinru Qiu, and Jian Tang. Dis- crete policy: Learning disentangled action space for multi-task robotic manipulation. arXiv preprint arXiv:2409. 18707, 2024.
[14] Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational sssionlanguaage-cttinn model for synergizing cogmition and action in robotic manipulation. arXiv preprint 11.iv::2411.196333 2024.
[15] Yichen Zhu, Zhicai Ou, Feifei Feng, and Jian Tang. Any2policy: Learning visuomotor policy with any-lttality. Advances in Neural Information Processing Syxtems, 37:133518-133540 2024.
[16] Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbecl. Any-p trajectory modeling for policy leamning arXiv preprint arXiv:2401.0002. 2023.
[17] Yichen Zhu Zhicai Ou, Xiaofeng Mou, and Jian Tang. netrieval-augmenne embodied agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17985- 17995, 2024.
[18] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for world control at scalc. arXiv preprint ar wxxi:2212.0688 2022.
[19] Soroush Nasiriany, Sean Kirmani, Tianli Ding. Laura Smith, Yuke Zhu, Danny Driess, Dorsa Sadigh, and Ted Xiao. Rt-affordance: Affordances are versatile intermediate representations for robot manipulation. 2024.
[20] Songming Liu, Lingxuan Wu, Bangguo Li. Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: 1b: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.0786/ 2024.
[21] Yueru Jia, Jiaming Liu, Sixiang Chen, Chenyang Gu, Zhilue Wang. Longzan Luo, Lily Lee, Pengwei Wang, Zhongyuan Wang, Renrui Zhang, et al. Lift3d foundation policy: Lifting 2d large-scale pretrained models for robust 3d robotic manipulation. arXiv preprint arXiv:2411.1862. 2024.
11

---

[22] Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju. Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. Robomind: Benchmark muliiemboddimnt intelligence normative data for robot manipulation. arXiv preprint urXiv:2412.13877 2024.
[23] Xiaogang Jia, Qian Wang, Atalay Donat, Bowen Xing, Ge Li, Hongyi Zhou, Onur Celik, Denis Blessing, Rudolf Lioutikov, and Gerhard Neumann. Mail: Improving imitation learning with selective state space models. In 8th Annual Conference on Robot Learning.
[24] Michat Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of- oftthhgghtthhttth reasoning, arXiv preprint arXiv:2407.0800 2024.
[25] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding. Danny Driess, Avinava Dubey, Chelsea Finn. et al. Rt-2: Aision-anguagg--ddtnn models transfer web knowledge robotic control. arXiv preprint arXiv:2307.1588
2023.
[26] Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing. Weinan Zhang. Huaping Liu, et al. Vision-language foundation models effective robot imitators, arXiv preprint xrXiv:2311.0131 2023.
[27] Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang. Qing Li, Song-Chun Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent in 3d world. In ICLR 2024 Workshop: How For Are We From AGI.
[28] Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng. Chaomin Shen, Yaxin Peng, et al. Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation. arXiv preprint ar 240: / /2514, 2024.
[29] Karl Pertsch, Kyle Stachowicz. Brian Ichter, Danny Driess, Suraj Nair. Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for iision-languge-acctio models. arXiv preprint XXix:5501.09747 2025.
[30] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian lchter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. no: A lision-language-action flow model for general robot control, 2024.
[31] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source lation-anguage-actton model. arXiv preprint urXiv:2406.09246 2024.
[32] Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.044 04133, 2023.
[33] Minjie Zhu, Yichen Zhu, Jinming Li, Junjie Wen, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifci Feng, et al. Scaling diffusion policy in transformer to billion parameters for robotic manipulation. arXiv preprint ar 24v: 14411, 2024.
[34] Yixiao Wang, Yifei Zhang, Mingxiao Huo, Ran Tian, Xiang Zhang, Yichen Xie, Chenfeng Xu, Pengliang Ji, Wei Zhan, Mingyu Ding. al. Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning. arXiv preprint arXiv:2407.00 240.201571.1.1 2024.
[35] Aaditya Prasad, Kevin Lin, Jimmy Wu, Linqi Zhou, and Jeannette Bohg. Consistency policy: Accelerated visuomotor policies via consistency distillation. arXiv preprint arXiv:2405.075: 2024.
[36] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement lcarning arXiv preprint arXiv:2305. 13301, 2023.
[37] Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Walke, Chelsea Finn, Aviral Kumar, and Sergey Levine. Zero-shot robotic manipulation with pretrained image-editing diffusion models, arXiv preprint MmXiv:2310.10639, 2023.
[38] Sudeop Dasari, Oier Mees, Sebastian Zhao, Mohan Kumar Srirama, and Sergey Levine. The ingredients for robotic diffusion transformers. arXiv preprint arXiv:2410. 10088, 2024.
[39] Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation, 2024.
12

---

[40] Moritz Reuss, Ómer Erding Yagmurlu, Fabian Wenzel, and Rudolf Lioutikov. Multimodal diffusion transformer: Learning versatile behavior from multimodal goals. 2024.
[41] Tony Z Zhao, Jonathan Tompson, Danny Driess, Pete Florence, Seyed Kamyar Seyed Ghasemipour. Chelsea Finn, and Ayzaan Wahid. Aloha unleashed: A simple recipe for robot dexterity. In 8th Annual Conference on Robot Learning.
[42] Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine. Fine-tuning of continuous-time diffusion models as regula--eg control. arXiv preprint Xiv:24422.15194, 2024.
[43] Masatoshi Uchara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Sergey Levine, and Tommaso Biancalani. Feedback efficient online fine-tuning of diffusion models. arXiv preprint arXiv:2402. 16359, 2024.
[44] Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, lean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna. Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20022 2025.
[45] Pengxiang Ding, Jianfei Ma. Xinyang Tong, Binghong Zou, Xinxin Luo, Yiguo Fan, Ting Wang. Hongchao Lu, Panzhong Mo, Jinxin Liu, et al. Humanoid-vla: Towards universal humanoid control with visual integration. arXiv preprint 255..:2502.14777 2025.
[46] Can Cui, Pengxiang Ding, Wenxuan Song, Shuanghao Bai, Xinyang Tong, Zirui Ge, Runze Suo, Wanqi Zhou, Yang Liu, Bofang Jia, et al. Openhelix: A short survey, empirical analysis, and open-source dual-system vla model for robotic manipulation. arXiv preprint arXiv:2505, 03912, 2025.
[47] Pengxiang Ding, Han Zhao, Wenjie Zhang, Wenxuan Song, Min Zhang, Siteng Huang. Ningxi Yang, and Donglin Wang. Quar-vla: Vision-language- Computer Vision, pages 352 367. Springer, 2024.
model for quadruped robots. In European Conference
[48] Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, et al. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-acttee model. arXiv preprint arXiv:2503. 10631, 2025.
[49] Qingwen Bu, Hongyang Li. Li Chen, Jisong Cai, Jia Zeng, Heming Cui, Maoqing Yao, and Yu Qiao. Towards synergistie, generalized, and efficient dual-system for robotic manipulation. arXiv preprint 1000022400.0800), 2024.
[50] Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Univla: Learning act anywhere with task-centric latent actions. arXiv preprint arXiv:2505.061 000.2250500611..0 2025.
[51] Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Pengju An, Xiaoqi Li, Kaichen Zhou, Senqiao Yang, Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Efficient vision- langon-llnnnggaattte acttinn model for robotic reasoning and manipulation. Advances in Neural Information Processing Syxtems, 37:40085 1440000851100 10, 2024.
[52] Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han. Shenzhi Wang, Shiji Song, Jiashi Feng, and Gao Huang. Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution. Advances in Neural Information Processing Systems, 37:56619-5666 2024.
[53] Shengliang Deng, Mi Yan, Songlin Wei, Haixin Ma, Yuxin Yang. Jiayi Chen, Zhiqi Zhang, Taoyu Yang, Xuheng Zhang, Heming Cui, et al. Graspvla: a grasping foundation model pre-trained on billion-ssale synthetic action data, arXiv preprint orXiv:2505.003: 033333, 2025.
[54] Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Minghan Li, Haoran Liu, Songlin Wei, Zhongyuan Wang. Zhizheng Zhang, and He Wang. Uni-navid: A video-based vision- saooo--lnnaag--cttinn model for unifying embodied navigation tasks. arXiv preprint arXiv:2412.066.
2024.
[55] Yuhui Chen, Shuai Tian, Shugao Liu, Yingting Zhou, Haoran Li, and Dongbin Zhao. Conrft: A reinforeed Ane-tuning method for vla models via consistency policy. arXiv preprint 2rx5v:2502.05450 2025.
[56] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fci Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. thain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
[57] Michal Zawalski. William Chen, Karl Pertsch. Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied ofain-of-thoughl reasoning, arXiv preprint arXiv:2407.08622 2024.
13

---

[58] Jinmiing Li, Yichen Zhu, Zhibin Tang, Junjie Wen. Minjie Zhu, Xiaoyu Liu, Chengmeng Li, Ran Cheng. Yaxin Peng. and Feifei Feng. Improving visionlanguage-actinn models via chain-of-afforddnce. arXiv preprint Xiix:24422224411.1 2024.
[59] Qingging Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, Cot-vla: Visual chain-of-thoupe -thoughhttt reasoning for asiion-angguageacce models. arxiv preprint urXiv:2503.22020, 2025.
[60] Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang. Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations. arXiv preprint arXiv:2412. 14803, 2024.
[61] Junjie Wen, Minjie Zhu, Yichen Zhu, Zhibin Tang, Jiaming Li, Zhongyi Zhou, Chengmeng Li, Xiaoyu Liu, Yaxin Peng, Chaomin Shen, et al. Diffusion-vla: Scaling robot foundation models via unified diffusion and autoregression. arXiv preprint ar arXi::2412.03293, 2024.
[62] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan. Jinze Bai, Keqin Chen. Xuejing Liu. Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing lision-lagguag model's 's perception of the world any resolution. arXiv preprint arXiv:2409. 2211:: 2024.
[63] Minjie Zhu, Yichen Zhu, Jinming Li, Junjie Wen, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifei Feng, Scaling diffusson policy in transformer to billion parameters for robotic manipulation. arXiv preprint arXiv:2409, 14411, 2024.
[64] Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Yu Wu, et al. Deepseckmoc: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv:2401.0600 06066, 2024.
[65]
Chengmeng Li, Junjie Wen, Yan Peng, Yaxin Peng, Feifei Feng, and Yichen Zhu. Pointvla: Injecting the 3d world into Dision-language-actionn models. arXiv preprint arXiv:2503.075/1 2025.
[66] Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang. Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GrOOt nl: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2533. 14734, 2025.
[67] Tsung- Yi Lin, Michael Maire, et al. Microsoft coco: Common objects in context. In Computer Vision ECCV 2014: 13th European Conference, Zurich, Swirzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[68] Amanpreet Singh, Vivek Natarajan, et al. Towards vqa models that can read. In Proceedings of the IEFF/CV conference on computer vision and pattern recognition, pages 8317-8326, 2019.
[69] Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700-6709, 2019.
[70] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Charles Xu, Jianlan Luo, Tobias Kreiman, You Liang Tan, Lawrence Yunliang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the abstract and introduction clearly state the claims we made, including the contributions made the paper and important assumptions and limitations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in a separate section in Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include theory assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: These details are clearly written in section Methodology and section Experi- ment.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:
[No]
Justification:
We are unable to release the robot data we collected due to policy restrictions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//uuuii wwwpc//ggue////eee CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
These details are clearly written section Experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: These details are clearly written in section Experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
They are discussed in the Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, the research conducted in the paper conform with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The image-text data we used has been cited and the name of license are included in Appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.paaersstt
package should be provided. For popular datasets, paperswi thcode. cww/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

Table 6: Ablation study on number of experts.
Table 7: Ablation study on reasoning-following enhancement module.
Expert numbers
Top-k numbers
OCR
Math
Method
Avg. success rate
3.58
1.73
Latter-half-layer injection
43/52
6
2.42
1.26
Full-layer injection
36/52
4
2
1.87
0.94
Former-half-laye injection
22/52
A
Limitation
Our work investigate to retain the pre-trained knowledge from the vision-language model in vision- language-action model. As such, the VLA are able to reasoning language instruction, and enforce the action model to follows such reasoning. Currently, we are unable fully retain the pre-trained knowledge from VLM. We observe that it inevitable that many capacity disappear during the fine-tuning with robot data. This is the most challenging part, and current approach cannot fully resolve this problem. We leave this the future work. Also, our current method is mainly conducted on table top tasks. We aim to expand the embodiment to mobile manipulator to perform more mongoooizon and complex real world tasks in the future.
the image observation and
B Implementation Details
B.1 Training details.
We utilize 8 NVIDIA H800 GPUs (80GB each) for training. We adopt mixed-precisio training (FP16) and use the Adam optimizer. For training stage I, we co-train on image-text data and robot data, setting the initial learning rate 2e-5 and training for 15k steps. For training stage 2, we freeze the VLM backbone. The model is trained for 50k steps, starting with a learning rate of 2e-5 and warm-up phase over the first 3k steps. In both stages, we apply cosine learning rate scheduler, scaling down the learning rate to 2e-6. The total training cost is 340 GPU hours.
B.2 Data details.
Image-text data composition. The image-text dataset used in our experiments integrates samples from multiple established benchmarks, including COCO, TextVQA, and GQA, alongside additional data specifically constructed to align with our task formulation. To ensure balanced representation, we incorporate approximately 32k samples from COCO, 20k from TextVQA, and 54k from GQA. These robotics-related image-text pairs employ the reasoning template used in the toy placement task, as illustrated in B.2 Furthermore, we utilize data from RoboPoint, comprising approximately 2k samples collected within a simulated environment. Although the RoboPoint data exhibits lower visual quality due to visual discrepancies and camera viewpoints, our experiments indicate that including this data enhances the disual-language alignment (VLA) model's spatial understanding capabilities. Additionally, we gathered 5k samples from wellworld environments, covering both tabletop setups and broader scenes. These samples follow similar annotation format to the LLaVA dataset, utilizing a question-answern structure. All collected data is combined and utilized collectively during training in our method.
Data pre-processing, For the image-text data, we limit each example to a maximum of 5 dialogue turns. If an instance originally contains more than 5 turns, retain the first turn and randomly sample four additional turns from the remainder. For the TextVQA dataset, we specifically select samples that do not contain numeric OCR tokens mathematical operators, our goal is to utilize pre-trained knowledge for open-world manipulation. We use the image resolution of 320 x 240.
Reasoning templates of robot data. All our robot data are annotated with sub-reasoning, similar to the approach used To.5 and Dex VLA. We initialize these reasoning annotations with fixed templates and then augment them using GPT-4o, following pipeline analogous to the one employed in training large language models. This method allows us to keep our reasoning phrase flexible, such that the action expert would not dominate by certain template.
22

---

C More Ablation Studies
We have discussed the importance of some key components our ChatVLA-2 in the main text, including the choice of mixture-of-expers and the two-stage training strategy. In this section, we will further discuss the following questions:
C.1 Ablation study on number of experts.
We conduct experiments to check how many experts we should use to better obtain pretrained knowledge from VLM while maintaining appropriate resource consumption. As is shown in Table 6 experimental results indicate that increasing both the total number of experts and the number of experts selected during inference can enhance the model's generalization ability in robotic scenarios.
possible explanation for this phenomenon is that, a limited number of experts tend to develop selection biases toward visually similar task images in such scenarios. This can lead to overfitting on robot data and result in the neglect of the pretrained VLM knowledge, ultimately degrading performance.
C.2 Ablation Study Layers for Injecting Reasoning-Following Enhancement Module.
As shown the main text, we replace the original observation embedding with reasoning tokens and use them condition the generation of scale and shift parameters in the latter half layers of the action expert. This mechanism effectively injected reasoning context into the model. In this section, conduct experiments on the place of injecting reasoning. The results shown in Table 7
Experiments show that the former half layers of action expert significantly impacts action generation stability. Introducing reasoning information into the former half layers actually increases instability in the generated actions, which turn significantly reduces task success rates. We hypothesize that this effect may due to our design choice of replacing the original observation embedding with reasoning information. One possible explanation is that the observations themselves may carry critical information for action generation, and their removal could negatively affect performance.
23


Reviews:
[{"review_id": "nfFh4GwyMX", "reviewer": "Reviewer_ow9p", "summary": "In this paper, it presents ChatVLA-2, a vision-language-action (VLA) model targeting open-world reasoning capability. The model aims to address a key limitation in current VLA methods—the loss of general reasoning capabilities from pretrained vision-language models (VLMs) when fine-tuned for robotic control. ChatVLA-2 incorporates a dynamic mixture-of-experts (MoE) backbone to separate multimodal understanding from robotic control, ensuring better retention of pretrained knowledge. A two-stage training process (co-training followed by action expert finetuning) aligns robot actions with high-level reasoning. The model is tested on a math-matching game and a toy placement task, demonstrating the effectiveness of preserving VLM’s pretrained knowledge through the proposed architecture and recipe.", "questions": "1. The description of reasoning-following module in line 186-187 is not clear to me. You first mention that you replace the observation embedding with reasoning tokens. What does this replace mean?\n2. The ablation of reasoning-following module is also not clear to me. First, the ablation is postponed to the appendix without any reference in the main text, while it is one of the two key contributions of the proposed architecture.  Second, why does the conditioning have to be in terms of adaptive layer norm. Could it simply be concatenated to the regular observation token sequences and let the self-attention of DiT module to figure out the reasoning tokens? (Or another way is through cross attention.)\n3. In terms of the training recipe, would a simple recipe where you simply freeze the VLM model weights at the beginning and only finetune DiT layers for one stage work well? The reasoning in the proposed experiments (summation & ocr) seems to be simple enough for a Qwen2-VL to zero-shot solve it. I'm actually confused why Table 4 with stage 2 only seems to achieve very low score on OCR & Math. Is the OCR & summation not solvable by Qwen2-VL?\n4. It's also not clear to me even if we decide to cotrain with reasoning data, why the we really need a separate stage II.  It seems to be a very weird design choice that you are training with the exact same objective, while dividing up into two stages. Is it because stage I is not trained long enough so you observed a lot of training instability with your VLM generated reasoning tokens?\n5. Also can the authors explicitly mention the difference between chatvla-2 vs. chatvla-1? I don't find this to be explicitly mentioned in the paper.", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strength**:\n1. The motivation to maximally preserve and leverage knowledge from pretrained VLM is clear, pinpointing the key bottleneck of current VLA model\n2. The paper is well written and easy to follow\n3. The design of the real robot experiment is good, sufficiently demonstrating the benefits of the proposed approach with a minimum set of experiments.\n\n**Weakness**\nWhile the paper tackles an important problem with good experimental results, it's not clear to me to some extent about why the proposed model architecture and training recipe contribute to the overall performance. See my questions below.", "was_revised": true, "final_justification": "This paper addresses a key limitation in current VLA methods—the loss of general reasoning capabilities from pretrained vision-language models (VLMs) when fine-tuned for robotic control. During rebuttal, the authors have addressed my questions. So I recommend acceptance for this paper.", "rebuttal": "### **Q1 What does the ‘replace’ in line 186-187 mean?**\n\nWe appreciate the reviewer's insightful question. In this context, “replace” refers to using the reasoning embedding instead of the observation embedding in the latter half of the transformer layers.\n\nSpecifically, we factorize the feature embedding of observation into multiple affine layers and integrate it into the transformer blocks. The Adaptive Layer Norm (AdaLN) block in the original ScaleDP can be formatted as\n\n$AdaLN_i(x) = (γ_i(t, o) + 1) · x + β_i(t, o), 1\\leq i\\leq N$\n\nwhere x is the input to the layer normalization, N is the number of layers, and γ(t, o) and β(t, o) are the adaptive scale and shift parameters regressed from the embedding vectors of timestep t and observation o.\n\nIn our ChatVLA-2, the AdaLN block is formatted as:\n\\begin{equation*}\nAdaLN_i(x) = \n\\begin{cases}\n(\\gamma_i(t, o) + 1) \\cdot x + \\beta_i(t, o), & \\text{if } i < \\frac{N}{2};\n\\\\\n(\\gamma_i(t, r) + 1) \\cdot x + \\beta_i(t, r), & \\text{if } i \\geq \\frac{N}{2}\n\\end{cases}\n\\end{equation*}\nwhere the original observation o is replaced to reasoning r in the latter half layers.\n\nWe will include these explanations in the revised version of our paper. We hope this can addresses some of your concerns.\n\n\n### **Q2 Why does the conditioning have to be in terms of adaptive layer norm? Could it simply be concatenated or another way through cross attention?**\n\nWe agree that alternative conditioning mechanisms merit thorough investigation. Our research, however, primarily focuses on improving the model's **reasoning-following capability. We propose the conditioning mechanism and demonstrate that such techniques can effectively enable the model to follow reasoning**, especially in open-world scenarios**.** We verified our claim through multiple challenging tasks. \n\nOther implementations (e.g., token concatenation or cross‑attention) are certainly possible, but they lie beyond the present scope. We plan to investigate such variants in future work and hope this addresses your concerns.\n\n### **Q3 Why ablation on stage 2 seems to achieve very low score on OCR & Math?**\n\nThank you for your valuable feedback. The low score for the \"stage-2 only\" setting is because it trains only on robotic data without freezing the Vision-Language Model (VLM). The primary objective of this ablation is to analyze **the significance of** **co-training with image-text data** (i.e., Stage 1) for preserving the VLM's capabilities under identical training conditions.\n\nWe apologize for the lack of clarity regarding the ablation setting. We will incorporate these explanations into the revised version of our paper. We hope this clarification addresses your concern, and we appreciate your thoughtful feedback.\n\n### **Q4 Why we really need a separate stage II? Is it because stage I is not trained long enough so you observed a lot of training instability with your VLM generated reasoning tokens?**\n\nWe would like to clarify that the training instability is not due to insufficient training in Stage I. Instead, it arises from the conflict training objective: next token prediction for the language component and noising-denoising for robot action generation.\n\nThis scenario mirrors multi-task learning, where diverse training objectives can lead to gradient conflicts, as illustrated in existing researches [1, 2, 3, 4]. Similar phenomena have also been reported in concurrent research [5] in robotics, that naive joint training with diffusion-based action experts significantly slows convergence and impairs language grounding.\n\nTherefore, Stage II training is essential. The objective of introducing Stage II training is to mitigate these gradient conflicts between the two tasks, thereby preserving the pre-trained knowledge within VLM and enabling it to effectively guide robot actions generation. \n\n### **Q5 The difference between our model vs. ChatVLA**\n\nChatVLA is a prior work that can do both multimodal understanding and robot manipulation in a unified model. However, it **fails to transfer its multimodal understanding capability to robot manipulation,** i.e., help robot to generalize on different task setup. In other words, even if ChatVLA can easily recognize text and mathematical symbols, while is able to complete math formulas when the user asks questions, it cannot turn their mind into physical action, such as pick the answer card on the board.\n\nThe main difference is that our proposed ChatVLA-2 can more **effectively leverage the knowledge** in real-world scenarios and transfer to a actionable movement, such as complete a math problem on a whiteboard. Our ChatVLA-2 demonstrates **stronger generalization under open-world scenarios,** representing a significant step toward the development of truly generalizable robotic foundation models.\n\n[1] Gradient Surgery for Multi-Task Learning, NeurIPS 2020\n\n[2] Conflict-Averse Gradient Descent for Multi-task Learning, NeurIPS 2021\n\n[3] RotoGrad: Gradient Homogenization in Multitask Learning, ICLR 2022\n\n[4] Recon: Reducing conflicting gradients from the root for multi-task learning, ICLR 2023\n\n[5] Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better, May 28, 2025"}, {"review_id": "yXp2CqnqbK", "reviewer": "Reviewer_chhG", "summary": "This paper introduces ChatVLA-2, a vision-language-action (VLA) model designed to preserve and leverage pretrained vision-language model (VLM) capabilities during downstream robotic fine-tuning. The core innovations are:\n\n\nA dynamic mixture-of-experts (MoE) architecture integrated into the VLM backbone to disentangle and selectively activate experts for multimodal understanding versus robotic control.\n\n\nA two-stage training strategy: (1) co-training on large-scale image-text and robot trajectory data to retain open-world reasoning, and (2) freezing the VLM and training only the action expert to align robotic actions with internal chain-of-thought reasoning.\n\n\nReal-robot experiments on two tasks, math matching and toy placement, demonstrating strong open-world OCR, mathematical, spatial reasoning, and action following, substantially outperforming baselines such as OpenVLA, DexVLA, and $\\\\pi_0$", "questions": "How exactly does the routing mechanism work? The paper mentions a dynamic MoE with 8 experts selecting 2 per input, but the actual gating function is never specified. Is this a learned linear layer, attention-based routing, or something else? What prevents the router from simply defaulting to the same expert pairs?\n\nWhat's the theoretical basis for expecting MoE to help here? Beyond the empirical results, is there any principled reason why mixture-of-experts should be particularly suited for this vision-language-action problem? The connection between expert specialization and catastrophic forgetting mitigation isn't obvious.\n\nWhy should we expect shared representations between VLM tasks and robot control? The claim that overlapping feature spaces exist seems central to the approach, but what's the intuition? Are low-level visual features really that transferable between, say, reading text and grasping objects?", "limitations": "Tasks remain relatively simple; real-world safety-critical applications may reveal additional failure modes.\n\nTraining large VLMs and MoE models incurs significant energy and carbon costs.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n• The dynamic MoE effectively mitigates catastrophic forgetting, preserving rich VLM knowledge (e.g., OCR, spatial reasoning, math) during robotic adaptation.\n• The two-stage training pipeline is well designed: joint co-training retains open-world competencies, and targeted action expert fine-tuning aligns reasoning with robot control.\n• Real-world robotic evaluations on a Franka arm across novel tasks provide strong empirical validation and clear performance gains over state-of-the-art methods.\n\nWeaknesses:\n\n1. Lack of formal metric for reasoning-action alignment: The paper claims that its reasoning-following enhancement module allows actions to follow internal reasoning. However, there is no quantitative measure to validate this claim. Let the model generate a reasoning sequence r and an action sequence a. A meaningful evaluation could define an alignment score such as:\n\n$A(r, a) = E_{r,a} [sim(r, f(a))]$\n\nwhere $f(a)$ converts action sequences into symbolic representations (e.g., \"pick number 3 from left\") and $sim$ is a semantic similarity function (e.g., BLEU score, cosine similarity in embedding space, or entailment score). However, such alignment is never measured, leaving the effectiveness of the core mechanism unverified. Furthermore, no behavioral consistency is assessed between reasoning errors and action failures, making it impossible to attribute improved performance to better reasoning-action coupling.\n\n2. MoE design lacks theoretical justification and introspection: The architecture uses a dynamic Mixture-of-Experts (MoE) with $N = 8$ experts and selects $K = 2$ per input. However, the routing policy is not mathematically specified, and no analysis is provided to examine the diversity or specialization of experts. For example, the routing entropy:\n\n$H(G(X)) = -\\\\sum_{i=1}^N G_i(x) \\\\log G_i(x)$\n\nwhere $G(x) \\\\in \\\\Delta^{N-1}$ is the softmax output over experts, could be used to evaluate whether experts specialize or overlap. Yet, this is never computed. Additionally, ablation experiments in Table 3 mix static, shared, and dynamic MoEs but without clarity on how these are implemented or how much capacity each configuration uses. The justification for choosing 8 experts and routing 2 is purely empirical and not supported by scaling laws, memory-accuracy tradeoffs, or comparisons with sparse/dense alternatives. This makes the MoE contribution under-theorized and difficult to reproduce or extend.", "was_revised": false, "final_justification": "", "rebuttal": "### **W1 The authors should converts action sequences into symbolic representations and use semantic similarity function to validate the reasoning-following enhancement module.**\n\nWe argue that constructing a function and using it to measure reasoning-action alignment is not suitable for the robotics domain and could lead to misleading conclusions. However, we agree with the reviewer that evaluating the reasoning-following module is essential. Therefore, we propose an alternative behavior-based evaluation strategy that incorporates module ablation. \n\n1. **Mapping actions to reasoning is problematic**. The suggested evaluation assumes a function maps low-level continuous robot actions to high-level discrete symbolic reasoning. We argue that this assumption is theoretically invalid in robotics. This is an ill-posed inverse problem, where solutions are not unique, stable, or guaranteed to exist. Meanwhile, low-level action data (continuous joint values) does not contain enough information to uniquely reconstruct the high-level symbolic reasoning, as is called “semantic gap”. \n2. We propose **an alternative evaluation** that does not require mapping actions to symbolic forms. Specifically, we design an intervention experiment in the math-matching game where we **swap the positions of the answer card** while keeping the math equation constant. **The math equation and the initial position of answer card are seen during training, while the distractor cards and the swapped position of answer card remain unseen.**\n    \n    This setup is designed to achieve two primary objectives. First, by using in-domain equations, we maximize the likelihood that the model generates correct reasoning. This allows our evaluation to focus exclusively on the model's reasoning-following module. Second, the introduction of unseen distractor cards and the dynamic change in the answer's position prevent the model from relying on pre-existing positional biases. Third, any observed performance difference observed before and after the card swap thus directly indicates the model’s reasoning-following capability .\n    We evaluate both the full ChatVLA-2 and the variant that removes the reasoning-following enhancement module under this setup. The results are presented below: \n    \n    | Setting | Success Rate (Before Swapped) | Success Rate (Swapped) |\n    | --- | --- | --- |\n    | Full Model | 12/13 | 10/13 |\n    | w/o Reasoning-Following Module | 11/13 | 2/13 |\n    \n    The results show that our full model maintains strong performance even after the card swap (10/13), indicating robust reasoning-following capability. In contrast, the ablated variant's performance drops sharply (from 11/13 to 2/13), highlighting its reliance on positional biases. This validates the effectiveness of the reasoning-following enhancement module.\n    \n\n### **W2-1 & Q1 The routing policy is not specified.  What prevents the router from simply defaulting to the same expert pairs?**\n\nFollowing the Noisy Top-K Gating mechanism[1] , our router uses linear layer with adaptive noise perturbation before taking the softmax function, which theoretically helps with load balancing. \n\n### **W2-2 No analysis is provided to examine the diversity or specialization of experts.**\n\nWe argue that we have made an ablation study on number of experts in the Appendix, Table 1, due to page limitation of the main paper. We also analyzed the possible explanation in the Appendix, Section D.1. We hope this can solve some of your concerns!\n\n### **W2-3 How Static, shared, and dynamic MoEs are implemented?**\n\nWe appreciate the reviewer's comment regarding the clarity of our ablation setting. We'd like to clarify the distinctions between static, dynamic, and shared MoE as used in our work.\n\n- Static MoE refers to an architecture where different data modalities are assigned to a specific expert via a fixed routing mechanism. Each data modality exclusively utilizes one predetermined expert.\n- Dynamic MoE, in contrast, employs a dynamic routing mechanism, allowing individual token—regardless of their modality—to be directed to various experts based on their representations.\n- Shared MoE extends dynamic MoE by incorporating an additional, universal \"shared\" expert that all data modalities traverse, alongside their dynamically chosen experts.\n\nWe will include these explanations in the revised version of our paper. \n\n### **Q2 Beyond the empirical results, is there any principled reason why mixture-of-experts should be particularly suited for this vision-language-action problem?**\n\nThe primary rationale for employing MoE stems from the fundamental challenge of **catastrophic forgetting** when developing Vision-Language-Action (VLA) models from pre-trained Vision-Language Models (VLMs). The significant distribution shift between high-level language and low-level robotic action causes severe cross-modal gradient conflict during fine-tuning, as p by continual learning theory. \n\nMoE directly addresses this interference. By dynamically routing inputs to a small subset of experts (parameter subsets), this architecture intrinsically fosters **parameter isolation** and **sparse activation**. These capabilities are fundamental to how MoE **mitigates catastrophic forgetting by limiting inter-task interference**.\n\nRecent theoretical work [2, 3, 4] rigorously supports this, proving MoE's specialization and sparsity intrinsically reduce interference. \n\nTherefore, MoE is a principled choice grounded in its inherent mechanism for isolating updates and protecting old knowledge, rigorously validated both theoretically and experimentally.\n\n### **Q3 Why shared representations between VLM tasks and robot control?**\n\nWhile multimodal understanding and robotic control operate in distinct output spaces, they fundamentally rely on processing shared **high-level semantic** representations (e.g., a door handle affords turning to open) and a unified understanding of environmental context semantics (e.g., the cup near a table edge demands careful action). \n\nIn particular, both VLM tasks require grounding in similar semantic concepts—such as spatial relations, object affordances, and scene geometry—to reason or act meaningfully. For example, understanding that a mug is typically grasped by its handle, or that only an empty plate can hold food, is crucial for both describing a scene and interacting with it.\n\nTherefore, leveraging these shared semantic representations is not only feasible but also a key pathway towards building more general and capable embodied agents.\n\n### **L1 Tasks remain relatively simple.**\n\nWe respectfully disagree with the reviewer’s assertion that these tasks are “simple.” They are demanding **open‑world problems** that require simultaneous generalization across visual perception, natural‑language instructions, multi‑step reasoning, and robotic control. In addition, each run is a **long‑horizon sequence**—our demonstration videos exceed two minutes—so the model must maintain coherent plans over hundreds of low‑level actions. To the best of our knowledge, no existing method—including OpenVLA, π0, π0.5, Gr00tN1, DexVLA, or the original ChatVLA—exhibits the scene understanding and cross‑domain generalization needed to solve such tasks without task‑specific training data.\n\nIf the reviewer is aware of VLA systems that successfully tackle comparable open‑world tasks by transferring knowledge from vision–language models to real‑world manipulation, we would greatly appreciate the citations and would welcome further discussion.\n\n### **L2 Real-world applications may reveal additional safety failures.**\n\nWe fully agree with the reviewer that real-world applications inherently present more unexpected failures than simulated environments, which is the reason why we conduct **all our experiments on real robots** (Franka and ARX R5). We did observe a certain failure rate as a result, as is shown in Section 4, Table 1 and 2.\n\nWhile the primary contribution focuses on advancing the model's reasoning and generalization capabilities in open-world settings, we recognize that further exploration of safety guarantees in diverse open-world scenarios is essential. We are looking forward to advancing this in future work.\n\n### **L3 MoE models incurs additional overhead**\n\nWe understand the reviewer's concerns regarding additional overhead. Compared to the dense 3B model, we observe an ~60% increase in inference latency and an ~40% increase in memory usage, primarily due to the increased number of active parameters as we use 8 experts with Top-2 gating. Given the substantial improvement in out-of-distribution task performance (success rate increased from near-zero to 82.7%) in open-world scenarios, **we consider this trade-off is justified and the overhead is acceptable.**\n\n[1] OUTRAGEOUSLY LARGE NEURAL NETWORKS: THE SPARSELY-GATED MIXTURE-OF-EXPERTS LAYER\n\n[2] Theoretical Analysis of Mixture-of-Experts in Continual Learning, ICLR 2025 Spotlight\n\n[3] Theory of mixture-of-experts for mobile edge computing\n\n[4] A comprehensive survey of mixture-of-experts: Algorithms, theory, and applications"}, {"review_id": "b3mfGxPrNB", "reviewer": "Reviewer_QrAS", "summary": "The paper presents ChatVLA-2, a vision-language-action model that retains pre-trained VLM knowledge for open-world robotic control. It uses a dynamic Mixture-of-Experts to disentangle conflicting features and a two-stage training strategy to preserve reasoning while enhancing action alignment. Experiments on math matching and toy placement show superior open-world generalization over frontier models, highlighting its ability to handle novel equations and spatial instructions. The work advances generalizable robotic foundation models by leveraging pre-trained VLM reasoning for real-world tasks.", "questions": "* Regarding baselines such as OpenVLA and DexVLA, do you fine-tune these models on your custom set of 900 trajectories? A comparison between their fine-tuned versions and the proposed ChatVLA2 would provide a more comprehensive evaluation.\n\n* Compared to dense models, MOE typically introduces higher inference costs. It would be helpful to analyze the additional inference overhead as well as the performance gains attributable to MoE. In addition, please provide the load balancing and expert specialization assessment of your trained MoE after fine-tuning.\n\n\n* The abstract mentions a three-stage training process, whereas Section 3 and the figures describe a two-stage training pipeline. Could you please clarify which description is accurate?", "limitations": "ChatVLA2 focuses on controlled tabletop manipulation scenarios to clearly disentangle and evaluate the model’s generalization to abstract reasoning tasks such as mathematics and spatial relations. A key direction for future work is to extend our architecture to more complex and dynamic domains, such as long-horizon procedural tasks or mobile manipulation settings that require tighter integration of navigation, interaction, and long-term memory.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "** Strengths**\n1. This writing of this paper is clear and easy to follow.\n2. The paper addresses a highly significant and critical problem in robotics: the erosion of pre-trained knowledge in Vision-Language-Action (VLA) models during fine-tuning. The authors clearly articulate this challenge and present their work in a well-structured and easy-to-follow manner. \n\n**Weaknesses**\n* Although the two tabletop pick-and-place tasks are well-designed, the authors do not fully represent the diverse challenges present in the VLA domain. I would like to see an evaluation of ChatVLA on a broader range of tasks, such as those involving navigation, long-horizon planning, or tool use, for example, tasks from CALVIN or RLBench. \n* Since one of the main contributions of this work is to retain the original knowledge and capabilities of the VLM, I suggest including additional results on multimodal reasoning and fine-grained visual recognition.", "was_revised": true, "final_justification": "The authors have provided satisfactory responses to my questions regarding the model's efficiency in MoE, its performance on general benchmarks, and its results on other long-horizon benchmarks.", "rebuttal": "### **W1 Authors should present evaluation on long-horizon tasks or simulation environments besides the two tabletop pick-and-place tasks.**\n\nThank you for raising these concerns. We address them point‑by‑point below.\n\n**1  Lack of suitable simulators**\n\nCurrent robotic simulators are tailored to short‑horizon pick‑and‑place or highly scripted scenes. None supports the **open‑world generalization and multi‑step reasoning** we study—namely, tasks that interleave vision, language, symbolic problem‑solving, and physical manipulation over several minutes. We therefore conducted real‑robot experiments to expose the full spectrum of perceptual ambiguities, occlusions, and physical uncertainties that a model must handle. If a simulator capable of this breadth exists, we would gladly adopt it and include the reviewer’s reference in a revision.\n\n**2  Long‑horizon difficulty—not “simple” pick‑and‑place**\n\nSome trial lasts **> 2 minutes** and comprises dozens of low‑level actions, multiple branching decisions, and recovery behaviors—not a one‑shot grasp. The robot must:\n\n1. **Parse instructions** stated in natural language.\n2. **Interpret visual cues** (e.g., handwritten formulas, answer cards).\n3. **Reason symbolically** to obtain a solution.\n4. **Execute and, if needed, correct** a manipulation plan in real time.\n\nWe also report **consecutive‑run success rates**, demonstrating that the behavior is reliable rather than cherry‑picked.\n\n **3 Why these tasks matter**\n\nThe experiments were crafted to showcase ChatVLA‑2’s ability to **translate high‑level reasoning into action**—for example, solving a math problem on a whiteboard and then physically selecting the correct answer card. Such “mundane” but cognitively rich activities reflect everyday human tasks far better than isolated pick‑and‑place benchmarks.\n\n### **W2 Including additional results on multimodal understanding and fine-grained visual recognition.**\n\nWe sincerely thank the reviewer for their thoughtful comments and valuable suggestions. We have conducted extensive evaluations across 12 diverse multi-modal understanding benchmarks, covering tasks such as document understanding (DocVQA), chart and scientific reasoning (ChartQA, AI2D), OCR-based question answering (TextVQA, OCRBench), and real-world fine-grained recognition (InfoVQA, RealWorldQA, MMStar). \n\nWe also present the results of baseline model ChatVLA, as is shown below:\n\n| Method | TextVQA | DocVQA | InfoVQA | ChartQA | MTVQA | AI2D | RealWorldQA | OCRBench | HallBench | MME | MMStar | MMMU |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| ChatVLA | 71.2 | 83.3 | 53.3 | 59.9 | 11.5 | 67.6 | 57.0 | 729 | 39.9 | 1435.2 | **47.2** | 37.4 |\n| **ChatVLA-2(Ours)** | **78.2** | **87.6** | **59.7** | **71.6** | **20.3** | **69.9** | **61.7** | **792** | **43.9** | **1464.5** | 42.3 | **38.8** |\n\nWhile ChatVLA adopts a static MoE design that explicitly routes VLM and robot data through separate experts, our method uses a dynamic MoE that jointly trains on both modalities with learned routing. The results demonstrate consistent improvements on 11 out of 12 benchmarks, with particularly notable gains in tasks requiring strong multi-modal understanding capabilities—such as +7.0 on TextVQA, +11.7 on ChartQA, and +6.4 on InfoVQA. These results indicate that key understanding abilities, including fine-grained recognition, OCR, and multimodal reasoning, are retained from the pre-trained VLM.\n\n### **Q1 A comparison between fine-tuned versions of OpenVLA , DexVLA and the proposed ChatVLA-2 would provide a more comprehensive evaluation.**\n\nWe would like to respectfully clarify that the baseline models compared in our paper were fine-tuned on robotic data of our tasks, rather than utilizing pre-trained open-source weights. This approach was taken to ensure a fair comparison, which we believe more clearly demonstrates the superior generalization capabilities of our model.\n\n### **Q2-1 Analyze the additional inference overhead**\n\nWe appreciate the reviewer's valuable feedback. Compared to the dense 3B model, we observe an **~60% increase in inference latency** and an **~40% increase in memory usage**, primarily due to the increased number of active parameters as we use 8 experts with Top-2 gating. Given the substantial improvement in out-of-distribution task performance (success rate increased from near-zero to 82.7%) in open-world scenarios, we consider this overhead acceptable.  ****Moreover, the inference speed of ChatVLA-2 is 30Hz, which is still faster than conventional VLA method like OpenVLA (5Hz). \n\n### **Q2-2 Provide the load balancing of MoE after fine-tuning.**\n\nWe've analyzed the expert usage statistics of our Mixture-of-Experts (MoE) model. The following data details the top-2 experts per layer and their corresponding token counts:\n\nLayer 0: Expert_3 - 981, Expert_5 - 883\n\nLayer 1: Expert_5 - 1129, Expert_4 - 1037\n\nLayer 2: Expert_7 - 1053, Expert_4 - 960\n\nLayer 3: Expert_4 - 912, Expert_2 - 779\n\nLayer 4: Expert_5 - 1047, Expert_2 - 730\n\nLayer 5: Expert_2 - 1051, Expert_3 - 886\n\nLayer 6: Expert_4 - 1097, Expert_1 - 747\n\nLayer 7: Expert_7 - 1157, Expert_2 - 569\n\nLayer 8: Expert_1 - 1187, Expert_7 - 657\n\nLayer 9: Expert_5 - 1002, Expert_0 - 928\n\nLayer10: Expert_0 - 1202, Expert_6 - 825\n\nLayer11: Expert_6 - 1124, Expert_1 - 1030\n\nLayer12: Expert_2 - 905, Expert_7 - 746\n\nLayer13: Expert_5 - 1168, Expert_0 - 733\n\nLayer14: Expert_4 - 889, Expert_2 - 720\n\nLayer15: Expert_2 - 1141, Expert_3 - 844\n\nLayer16: Expert_2 - 1099, Expert_0 - 891\n\nLayer17: Expert_6 - 1028, Expert_1 - 941\n\nLayer18: Expert_3 - 1273, Expert_7 - 904\n\nLayer19: Expert_7 - 745, Expert_5 - 733\n\nLayer20: Expert_4 - 1128, Expert_5 - 704\n\nLayer21: Expert_1 - 1195, Expert_3 - 755\n\nLayer22: Expert_6 - 1071, Expert_0 - 635\n\nLayer23: Expert_1 - 1102, Expert_6 - 832\n\nLayer24: Expert_5 - 1012, Expert_0 - 729\n\nLayer25: Expert_3 - 1065, Expert_4 - 936\n\nLayer26: Expert_4 - 1037, Expert_5 - 641\n\nLayer27: Expert_3 - 947, Expert_2 - 913\n\nFollowing the Noisy Top-K Gating mechanism[1] , our router uses linear layer with adaptive noise perturbation before taking the softmax function, which theoretically helps with load balancing. The average standard deviation of token counts per layer is approximately **282**, and the average top-1/top-8 expert token ratio is **5.27×**. The average entropy is **2.79**, compared to the theoretical maximum of 3.0.\n\nThese statistics provide an overview of expert utilization across our model, and the complete data is included in Table 1 at the end. We hope this can addresses some of your concerns.\n\n### **Q3 Typo: the “three-stage training process” in abstract**\n\nWe appreciate the reviewer's careful attention to detail. The methodology employed is the **two-stage training process** consistently described throughout the main body of the paper. We sincerely apologize for any confusion this oversight may have caused and will rectify it in the final version of the manuscript.\n\n\n[1] OUTRAGEOUSLY LARGE NEURAL NETWORKS: THE SPARSELY-GATED MIXTURE-OF-EXPERTS LAYER\n\n**Table 1 Expert Traffic per Layer**\n| Layer\\Expert | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| 0 | 384 | 735 | 434 | 981 | 151 | 883 | 332 | 228 |\n| 1 | 572 | 415 | 249 | 238 | 1037 | 1129 | 198 | 290 |\n| 2 | 216 | 546 | 771 | 186 | 960 | 190 | 206 | 1053 |\n| 3 | 147 | 241 | 779 | 773 | 912 | 718 | 269 | 289 |\n| 4 | 363 | 452 | 730 | 534 | 189 | 1047 | 310 | 503 |\n| 5 | 351 | 311 | 1051 | 886 | 712 | 412 | 215 | 190 |\n| 6 | 377 | 747 | 530 | 315 | 1097 | 401 | 476 | 185 |\n| 7 | 518 | 464 | 569 | 422 | 369 | 296 | 333 | 1157 |\n| 8 | 248 | 1187 | 186 | 425 | 568 | 563 | 294 | 657 |\n| 9 | 928 | 217 | 365 | 580 | 426 | 1002 | 276 | 334 |\n| 10 | 1202 | 332 | 470 | 235 | 626 | 228 | 825 | 210 |\n| 11 | 675 | 1030 | 200 | 373 | 223 | 245 | 1124 | 258 |\n| 12 | 606 | 475 | 905 | 434 | 271 | 336 | 355 | 746 |\n| 13 | 733 | 181 | 527 | 321 | 540 | 1168 | 203 | 455 |\n| 14 | 540 | 584 | 720 | 282 | 889 | 369 | 235 | 509 |\n| 15 | 197 | 281 | 1141 | 844 | 305 | 615 | 435 | 310 |\n| 16 | 891 | 394 | 1099 | 383 | 621 | 348 | 198 | 194 |\n| 17 | 338 | 941 | 475 | 323 | 176 | 540 | 1028 | 307 |\n| 18 | 357 | 357 | 197 | 1273 | 544 | 266 | 230 | 904 |\n| 19 | 428 | 431 | 487 | 489 | 303 | 733 | 512 | 745 |\n| 20 | 491 | 557 | 233 | 307 | 1128 | 704 | 319 | 389 |\n| 21 | 284 | 1195 | 272 | 755 | 322 | 469 | 629 | 202 |\n| 22 | 635 | 259 | 592 | 470 | 487 | 401 | 1071 | 213 |\n| 23 | 447 | 1102 | 497 | 328 | 480 | 184 | 832 | 258 |\n| 24 | 729 | 354 | 429 | 417 | 348 | 1012 | 571 | 268 |\n| 25 | 193 | 610 | 263 | 1065 | 936 | 510 | 276 | 275 |\n| 26 | 253 | 553 | 563 | 624 | 1037 | 641 | 270 | 187 |\n| 27 | 291 | 503 | 913 | 947 | 248 | 370 | 446 | 410 |"}, {"review_id": "ot0cxljkK9", "reviewer": "Reviewer_XWQW", "summary": "The paper is a further extension of building VLA (vision-language-action) models. This method, dubbed ChatVLA-2, draws a distinction from the prior ChatVLA-1 model by putting more focus on open-world reasoning.\n\nThe general VLA framework (popularized by OpenVLA, $\\pi_0$ and $\\pi_{0.5}$, DexVLA, etc.) is to take a pretrained vision-language model, and finetune it on robot-specific data. Sometimes this finetunes the entire model, and sometimes it finetunes a smaller subset of params. This work is more like the second.\n\nThe prior ChatVLA-1 model ran 2 stage training on a mixture-of-expert LLM backbone. A robot-control-only expert was first trained in isolation with a fixed VLM for general visual understanding. Then, after this control expert was trained enough, the entire model was trained by co-training on robotics + vision datasets, to try to integrate the control expert with the remaining experts.\n\nChatVLA-2 updates this by no longer using a static control-only expert. Instead, it hews closer to the Qwen2 architecture they finetune, where. the model has multiple experts (8 of them), of which 2 are selected via a learned gating network, and we rely on the gating network to segregate robot features from general vision understanding features. The paper argues this makes it easier for the model to share representations between the robot tasks and vision tasks, compared to static experts.\n\nHowever, in this setup, there is still a seperate robot-specific model: an action model, in this case a 1B diffusion policy, which takes the LLM output and generates the final actions. (The paper calls this an \"action expert\" but IMO this is confusing because it is not an expert in the MoE of the LLM. Rather it is an entirely separate model.) Training of ChatVLA-2 is inverted from the previous model, where instead of finetuning on robot data then robot + vision data, we first finetune on robot + vision data then robot data only. This is done because the Dynamic MoE needs to learn the shared robot + vision representations, before the action expert can refine them afterwards. (This setup is essentially the same as DexVLA, which also used Qwen2-VL + a 1B diffusion action policy, although DexVLA did not use MoE)\n\nThe distinction from DexVLA is how the open world understanding is used to condition the diffusion model. Conditioning is done via FiLM generating scaling + offset params for each layer of the policy. In DexVLA, conditioning information for the diffusion policy is generated from the image observation + language instruction. In ChatVLA-2, it is instead generated from generated language reasoning tokens, which take the place of the image observation. Importantly this is only for the last half of the model: the first half of the diffusion policy sees FiLM params from the original image instead.\n\nThis is evaluated on tasks intended to test the reasoning abilities of the model: a math task where the model needs to move the object of matching number, and a visual task where the robot needs to place a toy relative to another object.\nPrevious VLAs (OpenVLA, pi0, DexVLA) have issue where pretrained knowledge can be eroded by the robotics finetuning.\n\nResults for ChatVLA-2 are better on open-world evaluation settings, compared to DexVLA and ChatVLA which fit in-domain data well but generalize more poorly elsewhere.", "questions": "I am curious if the authors ever did experiments on forcing the correct reasoning - always including the correct digits, object bounding boxes, etc. in the open-world reasoning section. This seems like a better way to explore the best way to incorporate that reasoning.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Modifying the generation of the conditioning information to include higher level language reasoning rather than using just the image makes sense for tasks where you expect the important information for the task to not necessarily be easily extractably from the high-level instruction and image.\n\nMy main complaint of the paper is that most of the failings of the prior models are tied to poor OCR and poor object recognition respectively. The OCR score is out of 4: 1 for reading the equation numbers, 1 for reading the printed numbers on cards, 2 for identifying the right math operation. And both DexVLA and ChatVLA are awful at identifying this, which makes their success rates low. But this is more an indictment of the VLM's ability to identify the numbers than an indictment of the diffusion policy's ability to execute based on the identified numbers. It does not feel like a test of the visual reasoning module. A similar argument can be made about the toy placement task, where the bottleneck is on identifying the spatial affordance, before any of the diffusion conditioning occurs.\n\nAside from this, the results do support the benefits of building an MoE architecture with a dynamic gating network, compared to the dense models common in prior work, and the ablations of where language reasoning tokens are applied in FiLM do show that important information is carried in those tokens.\n\nNote: demo video is not visible from provided link, needed to download repository manually.", "was_revised": true, "final_justification": "After reading other reviews, I find my mind is not changed about the paper. I had some issues with results I would have liked to see to make the paper more convincing, but authors have clarified that some of the ablations I had in mind would not be feasible in the current model architecture.", "rebuttal": "### **W1 The failures of prior models are primarily attributed to perception weaknesses, rather than deficiencies in the diffusion policy’s execution ability.**\n\nTraditional end-to-end approaches, due to their tightly coupled internal processes, often make it challenging to attribute failures to perception, reasoning, or execution.\n\nEven methods incorporating intermediate language outputs (\"reasoning\") can encounter this challenge. The core issue is the need for **reliable mappings** between perception and reasoning, and between reasoning and execution. Without guarantees on the correctness of these reasoning steps or explicit modules ensuring actions follow the reasoning, definitive failure attribution remains problematic. \n\nIn contrast, our proposed method addresses these limitations. By employing a Mixture of Experts (MoE) architecture, our approach effectively preserves the knowledge from pre-trained Vision-Language Models (VLMs) to enable robust reasoning. Furthermore, our reasoning-following module and two-stage training recipe make our model's actions consistent with its internal reasoning.\n\nGiven these considerations, we suggest that attributing failures of prior models solely to perception weaknesses is not appropriate.\n\n### **Q1 Experiments on forcing the correct reasoning is valuable.**\n\nWe appreciate the reviewer's valuable feedback. We would like to clarify that it is infeasible for our model to explicitly enforcing correct reasoning during inference. This is because the reasoning output (e.g., textually stated digits or bounding boxes) is decoded directly from the hidden state, while the policy head takes these same hidden states—rather than explicit reasoning representations—as its input.\n\nGiven this design, it is infeasible to externally impose specific reasoning components on the model during the inference phase. We appreciate the reviewer's understanding of this architectural consideration.\n\n### **Summary1 The paper calls this an \"action expert\" but this is confusing because it is not an expert in the MoE of the LLM.**\n\nWe appreciate the reviewer's comment. To clarify, our \"action expert\" is an **independent action output module**, not an expert within an MoE framework. This terminology is consistent with its use in previous works like pi0, pi05, pi05-KI, and DexVLA.\n\nWe will include these explanations in the revised version of our paper. We hope this can addresses some of your concerns."}]

Author rebuttal:

Meta-review: This paper introduces a new VLA architecture - chatVLA-2, which aims to retain the open world reasoning capabilities of VLMs in VLA models, which are often lost during action prediction finetuning. The two major changes that chatVLA-2 makes are - 1) a switched a dynamic MoE backbone that helps alleviate catastrophic forgetting, 2) 2 stage training with co-training, followed by action finetuning lets action prediction be aligned with reasoning.  This methodology is then validated on two real world tasks - math matching and toy placement, which test whether the original VLMs reasoning capabilities are retained and combined with action prediction capabilities. 

The reviewers generally appreciated the motivation and execution of the paper, but raised some valid concerns. Specifically, more explanation of *why* these design choices make such a difference, some additional baselines with simpler action-reasoning alignment recipes, validation on standard multimodal VLM prediction tasks, more formal description and validation of the claims (as suggested by reviewer chhG), some additional experiments on higher dexterity problems. These should be updated in the final vesion of the manuscript. 

Overall the premise of the paper is valuable and it adds a useful datapoint to the literature on VLA models.
Decision: Accept (poster)