BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models
Authors: Peiyan Li
Yixiang Chen
Hongtao Wu
Xiao Ma
Xiangnan Wu
Yan Huang
Liang Wang
Tao Kong
Tieniu Tan
Venue: NeurIPS 2025 poster ()
Keywords: 3D Robot Manipulation；Vision-Language Model；Vision-Language-Action Model；
Primary area: reinforcement_learning
TL;DR: We propose a 3D VLA model that aligns the input and output within a shared 2D space in both pre-training and fine-tuning which enables high data efficiency and achieves impressive performance in both basic and generalization settings.

Abstract: Recently, leveraging pre-trained vision-language models (VLMs) for building vision-language-action (VLA) models has emerged as a promising approach to effective robot manipulation learning. However, only few methods incorporate 3D signals into VLMs for action prediction, and they do not fully leverage the spatial structure inherent in 3D data, leading to low data efficiency. In this paper, we introduce a new paradigm for constructing 3D VLAs. Specifically, we first pre-train the VLM backbone to take 2D images as input and produce 2D heatmaps as output. Using this pre-trained VLM as the backbone, we then fine-tune the entire VLA model while maintaining alignment between inputs and outputs by: (1) projecting raw point cloud inputs into multi-view images, and (2) predicting heatmaps before generating the final action. Extensive experiments show that the resulting model, BridgeVLA, can learn 3D manipulation both efficiently and effectively. BridgeVLA outperforms state-of-the-art baselines across three simulation benchmarks. In RLBench, it improves the average success rate from 81.4\% to 88.2\%. In COLOSSEUM, it demonstrates significantly better performance in challenging generalization settings, boosting the average success rate from  56.7\% to 64.0\%. In GemBench, it surpasses all the comparing baseline methods in terms of average success rate. In real-robot experiments, BridgeVLA outperforms a state-of-the-art baseline method by 32\% on average. It generalizes robustly in multiple out-of-distribution settings, including visual disturbances and unseen instructions. Remarkably, it is able to achieve a success rate of 95.4\% on 10+ tasks with only 3 trajectories per task, while other VLA methods such as $\pi_{0}$ fail completely. Project Website: https://bridgevla.github.io/.

BridgeVIs VLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models
Peiyan Li Yixiang Chen Hongtao Wu? Xiao Ma Xiangnan Wu!
Yan Huang 3 Liang Wang Tao Kong², Tieniu Tan 13,55
NNw Laboratory of Pattern Recognition (NLPR).
Institute of Automation, Chinese Academy Sciences
2 ByteDance Seed School of Artificial Intelligence, University of Chinese Academy of Sciences FiveAges Nanjing University
Abstract
Recently, leveraging pre-trained vision-languee models (VLMs) for building iision-language-attionn (VLA) models has emerged as a promising approach to effective robot manipulation learning. However, only few methods incorporate 3D signals into VLMs for action prediction, and they do not fully leverage the spatial structure inherent in 3D data, leading to low data efficiency. In this paper, we introduce a new paradigm for constructing 3D VLAs. Specifically, we first pre-train the VLM backbone take 2D images input and produce 2D heatmaps as output. Using this pre-trained VLM as the backbone, we then fine-tune the entire VIA model while maintaining alignment between inputs and outputs by: (1) projecting raw point cloud inputs into multi-view images, and (2) predicting heatmaps before generating the final action. Extensive experiments show that the resulting model, Bridge can learn 3D manipulation both efficiently and effectively. Bri outperforms state-of-the-art baselines across three simula- tion benchmarks. In RLBench, it improves the average success rate from 81.4% 88.2%. In COLOSSEUM, demonstrates significantly better performance challenging generalization settings, boosting the average success rate from 56.7% 7% 64.0%. GemBench, surpasses all the comparing baseline methods terms of average success rate. In real-robot experiments, BridgeVI.A outperforms a state- of-the-art baseline method by 32% on average. generalizes robustly in multiple out-of-distribution settings, including visual disturbances and unseen instructions. Remarkably, it is able to achieve a success rate of 95.4% on 10+ tasks with only 3 trajectories per task, while other VLA methods such as fail completely. Project Website: httph: /tbrid//bbrideella wwwpeeelllggttt.......e github.ion
1 Introduction
Leveraging pre-trained vision-language models (VLMs) 3 43 24 for developing large vision- language-action (VLA) models has become a promising method for learning generalizable and robust manipulation policies [26 4 17 [1 7. However, most VLA models only incorporate 2D image inputs and require extensive efforts on data collection. On the other hand, 3D robot policies leverage 3D structural priors in model design and demonstrate exceptional sample efficiency in learning complex 3D robot manipulation tasks [39] 25 15) Can we develop a unified 3D VLA model which combines the effectiveness of VLA models with the efficiency from 3D policies?
Project lead Conrespnnding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Previous framework
D-00-00-6
Previous 3D-VLA
VLM
--0-0
D-00-0
"hndd al Instwnoee
That Uieittng marron spous
2D Finetune
2D Heatmap
2D Protrain
Projection
actions
Our framework
BridgeVLA
Image
Instructions
Simulation
Bashaap Upsamp
Reshapa Upsample
VLM
>25> >yp>
BridgeVLA
Generalization Trajectories
RLBanch
Colosseum
5-0--0
0-00-0
"Put the ring. spolee"
Background
Lighting
Instances of fock**
DM
Text Token
2D Image
3D Scorne
1D Fexture
Combinaton
Height
Cetegory
Token
Token
Token
Figure 1: Overview. BridgeVLA is novel 3D VLA model that aligns the input and output within unified 2D image space. It is pre-trained on object grounding using 2D heatmaps and fine-tuned on action prediction for 3D manipulation. Experiment results in both simulation and the real world show that it is able to learn 3D manipulation both efficiently and effectively.
Although there have been some works exploring integrating 3D information into VLMs for developing 3D VLA models [52 37 these works typically convert actions into token sequences that do not have spatial structure and use next-token prediction to predict actions. This strategy fails to take advantage of the 3D structural priors as previous efficient 3D policies [39 25 15 that align the observation input and action output into unified space, therefore leading to poor sample efficiency. Another significant challenge in developing 3D VLA models lies the misalignment between the 3D inputs used in action fine-tuning and the 2D image inputs used in original VLM pre-training, causing large distributional shift from the original VLM pre-training.
To tackle the challenges mentioned above, as inllustrated in Fig. 1 we present Bridge VLA, a novel 3D VLA model that achieves remarkable sample efficiency and strong generalization capabilities. To
ensure input alignment with the pre-trained VLM backbone, Bridge transforms a 3D point cloud
observation into multiple 2D images captured from different orthographic projection views [4 [5].
To leverage the structural priors of the 3D input, Bridge VLA is trained to predict 2D heatmaps for translational action prediction. The 2D heatmaps, generated from the tokens corresponding the projection images, share the same resolution as these images, aligning the input observations and output actions within a unified spatial structure. Given that the original VLM is pre-trained to predict token sequences, which is incompatible with our VLA's 2D heatmap output, we also introduce scalable pre- pre-training method, which trains the model to ground objects with heatmaps conditioned on text inputs. This pre-training method equips the VLM with the capabilities to predict heatmaps before downstream fine-tuning for policy learning. Overall, our design aligns the input and output within shared 2D space in both pre-training and fine-tuning.
We perform extensive experiments in both simulation and the real world evaluate the proposed method. Results show that Bridge VLA is able to learn 3D manipulation both efficiently and effec- tively. It outperforms state-of-the art baseline methods RLBench [9) improving the average success rate from 81.4% to 88.2%. In COLOSSEUM 35 it showcases strong performance in chal- lenging generalization settings, boosting the success rate from 56.7% to 64.0%. In GemBench [[2], it surpasses all the comparing baseline methods in terms of average success rate. In real-robot experi- ments, we evaluate on seven different settings, spanning from visual perturbations to manipulating objects from unseen categories. VrA surpasses a state-of-the-arr method by 32% on average and demonstrates strong performance generalizing to multiple distr--dittiiuttion settings. Notably, Bridge VLA is able to achieve a success rate of 96.8% 10+ tasks using only 3 trajectories per task

---

for training, highlighting its superb sample efficiency. In summary, the contributions of this paper are threefold:
We introduce BridgeVLA, novel 3D VLA model that efficiently and effectively learns 3D robot manipulation with a vision-language model via input-output alignment with 2D heatmaps.
We propose scalable pre-training method to equip the model with the capability to predict heatmaps conditioned text inputs via object grounding.
We conduct extensive experiments in both simulation and real-world environments to thoroughly evaluate the proposed method. Results show that BridgeVLA outperforms state- of-the-art methods in both settings and achieves exceptional sample efficiency in real-robot experiments.
2 Related Work
Language- Condiiioond Visuomotor Policies. Most canguage-conditioned visuomotor policies
employ transformers to process 2D visual inputs and directly generate 3D) actions for manipulation [6]
26
4 17,
10,
31 30
8
51 44 In these works, leveraging pre-trained vision-language models
(VLMs) for developing large vision- languagg-atttiooonn (VLA) models has become popular for its
effectiveness on learning complex manipulation 7 26 31 17]. However, such 2D image-based
policies typically require significant efforts on data collection, often needing hundreds of trajectories per task to learn effectively. On the other hand, 3D manipulation policies hold great potential for efficient learning by taking advantage of the spatial structure inherent in the 3D inputs. A popular [3] proposes to create 3D feature cloud by lifting image features to the observation point cloud and predicts translational actions via classification for 3D points in the observation space. Another line of works utilize voxels to represent the observation space and predict translational actions within the voxel space, unifying the input observation and output actions within the same space (39 20). Recently,
line of works take as inputs point cloud data p 48 47 B 25]. For example, Act3D
RVT [14] and RVT-2 [1] propose to leverage orthographic projection of 3D point clouds to convert 3D signals to 2D images to avoid high computational cost on processing 3D inputs. Different from the above methods, our method aims to unify the effectiveness of VLA models and the efficiency of 3D policies within a single cohesive framework, combining the best of both worlds.
(VLA) Models. While 2D VLA models have been extensively remain relatively under-explored. Zhen et al. [52 build 3D-VLA on top of a large language model (LLM) and train the model to perform 3D reasoning, multi-modal goal generation, and robot planning. Lift3D [22] proposes to enhance 2D foundation models (e.g., DINOv2 (34) with implicit and explicit 3D robotic representation for learning 3D manipulation policies. FP3 47 leverages a transformer to fuse the information from point clouds, proprioceptive states, and language instructions. PointVLA 29 utilizes a VLM and a point cloud encoder to process 2D images and 3D point clouds, respectively. The embeddings from both encoders injected into action expert for action prediction. SpL (37) introduces Ego3D position encoding to inject 3D information into 2D image observation and adaptive action grids to represent robot movement
3D Vision-Language-al studied, 3D VLA models [52] 22 47 29
more transferable way. Our method is different from the above methods in that it is designed in a way to take advantage of the spatial structure of 3D inputs in action prediction. In addition, bridges the gap between the 2D image inputs of pre-trained VLMs and the 3D inputs by projecting the 3D inputs into multiple 2D images instead of injecting 3D information into the VLMs. Such design enables to simultaneously leverages the broad knowledge in the VLM backbone and the spatial structure priors embedded in 3D inputs.
3
Bridge VLA
3.1 Preliminaries
Bridge VLA aims to learn a multi-task 3D robot manipulation policy ग,, which maps the observation and language instruction to an action a:
(o, 1)
(I)
3

---

2D Heatmap Pre-training
*
Weights
Preprpmnen
SigLIP
Upsemple
Unfreezz Wetghts
Germia
Imaage
insianciss of Fradee
2D Detection
Token
Dara
Capy We grit
Weights
3D Action Fine-tuning
Orthographic Propection
* SigLIP
Rearrange
Convex
Upsamplle
"Put the ring meroon spoie
3D Point Coud Deta
Tokenizer
MLP
Gripper
Coilbtio
Figure 2: Model Architecture. (a) 2D Heatmap Pre-training: train BridgeVLL VLA 2D object detection datasets. The model takes as imputs an image and a language describing the target object and outputs 2D heatmap which highlights regions of interest that correspond to the target object. Note that the bounding box shown here is for illustrative purposes only; it is not present in the image when input to the model. (b) 3D Action Fine-tuning: the model takes inputs three orthographic projection images of 3D point cloud and language instruction. outputs three 2D heatmaps, which highlight the position of the end-effector the next keyframe across all three views. For the remaining action components, it uses an MLP to process the image feature tokens to predict the rotation action, gripper action, and collision flag of the next keyframe.
We assume access to set of expert demonstrations = N containing N trajectorics. And each trajectory contains language instruction and a sequence of observation-action pairs, i.e., r' = {l', (oj. af).... (o a The observation o is one or multiple RGB-D images captured from multiple viewpoints. Following prior works [39 14 13 the action consists of 6-DoF end-effector pose E (3),). a target gripper state E {0, 1}. and a collision flag c E {0, 1} of the next key frame. The collision flag indicates whether the motion planner should avoid collisions while moving towards the target pose. key frame typically captures important or bottleneck steps Bridge operates through iterative process: 1) predicting the action a, conditioned on the current observation and instruction 2) moving to the predicted next keyframe pose using sampling-basee motion planner [40] 28 [T]. 3) updating
in trajectory (detailed in appendix B.3
2X
observation and repeating until task completion or reaching maximum step H
As illustrated in Fig. BridgeVLA employs dual-phase training recipe. During pre- pre-training, is trained to predict 2D heatmaps on object detection datasets. During fine-tuning, point clouds are projected into multiple 2D images as inputs to the VLM backbone. The model is trained to predict 2D heatmaps for estimating the translational action and other action components. This design aligns the input and output within shared 2D space in both pre-training and fine-tuning.
3.2 2D-Heatmap Pre-training
The VLM backbone was originally pre-trained to predict token sequences without spatial structure. To equip it with the same ability predict heatmaps downstream policy learning. introduce a pre-training stage which trains the model to ground target objects via heatmaps. Concretely, we leverage the 120K object detection split of RoboPoint [4) as our pre-training dataset. For each image, we construct the ground-truth heatmap H from the bounding boxes of all objects of interest. Specifically, for each object, construct probability map with spatial truncation:
H HF*(x) (x) = (x(x) if px Pain otherwise
where x (u, v) denotes the pixel position, (x) exp (-I (1122=)) x; is the center the object bounding box, and Pmin is probability threshold. For all the objects of interest, fuse the

---

probability map of all objects via averaging and normalization to obtain H
Havg(x) where Havg(x) = H (x)
NV
H (x) = Exen-Havg Havg(x)
N
where Q denotes the pixel space. Please refer to Fig. 9 for samples of the ground-truth heatmaps.
As illustrated in Fig. 2 we input an image along with the text prompt describing the objects of interest into the VLM backbone of BridgeVLA. In this paper, we employ PaliGemma [3] as the VLM backbone, which consists of a SigL.P vision encoder (50) and Gemma transformer backbone [4]] During its pre-training, PaliGemma takes as input one multiple 2D images together with prefix text (e.g,a question about the image) and outputs a suffix text (e.g. an answer to the question). While the model uses causal attention for predicting suffix text tokens, it adopts bidirectional attention for the image tokens and the prefix text tokens. This allows the image tokens to fuse information from the prefix text.
To predict the heatmap, we first rearrange the output image tokens according their patch positions to reconstruct the spatial feature grid. A convex upsampling block [42] then converts the grid into a heatmap with the same resolution as the input image. Unlike fixed methods (e.g., bilinear or nearest-neighbor, this upsampling module learns pixel-wise interpolation weights, allowing for finer spatial detail recovery. The whole pipeline is trained with a cross-entropy loss to predict heatmaps that localize the position of all objects of interest in the image. We emphasize that the proposed pre-training strategy outputs a spatially aware 2D heatmap, in contrast to the conventional next-token- prediction used in prior works [52] 33). Moreover, this approach is highly scalable, as it can, in principle, leverage any vision-language datasets that can be formulated as a heatmap prediction tasks, such as keypoint detection and semantic segmentation.
3.3
3D Action Fine-tuning
During fine-tuning, we first reconstruct a point cloud of the scene from the RGB-D images captured from calibrated cameras. To align with the 2D image input of the VLM backbone, we render three orthographic projection images of the point cloud from three viewpoints (top, front, and right) and use these images as the input images for the VIM backbone as in RVT [14] and RVT-2 [15] These images, along with the task instruction, are then fed into the pre-trained VLM backbone to
generate a heatmap for each of the three views. Importantly, we do not incorporate any additional information (e.g., robot states) during the VLM forward pass to minimize the distribution shift between pre-training and fine-tuning.
For translational actions, we back-project the heatmaps of all three views to estimate the scores of all 3D point grids distributed uniformly across the robot workspace. The position of the 3D point with the highest score determines the end-effector's translation in the next keyframe. Similar to previous works [14 15), we use Euler angles to represent rotational actions where each axis is discretized into 72 bins. To predict the rotation, binary gripper action, and collision avoidance flag, we integrate features from global and local contexts. For the global feature, max-pooling is applied to the output tokens of each inputted orthographic projection image, resulting in three tokens in total one for each view. For the local feature, extract a token from the heatmap peak of each view, also resulting in three tokens in total. All these tokens are concatenated and passed through MLP to predict the rotation action, gripper action, and collision avoidance flag.
Following the approach prior works (20 15), BridgeVLA adopts coarse-to-fin refinement strategy for accurate action prediction. After the initial prediction on the original point cloud, we zoom in and crop the point cloud with a cuboid centered at the predicted translation. A second forward pass is performed on the cropped, zoomed-in point cloud. The predicted action from the second pass used for execution.
The training loss during fine-tuning consists of four components:
L = Ltrans + Lrot
Lgripper + Lcollision
(4)
Similar to pre-training, Livans cross-entropy loss that supervises the heatmap prediction for translational actions. The ground-truth heatmap for each orthographic view is the normalized single- object probability map defined in Eq. 2 where x; represents the projected pixel position of the ground-truth end-effector position in the next keyframe. As we discretize the Euler angles for rotation
5

---

Overall
Task Success Rute (%)
Models
SR (%)
Avg.
Avg. Rank
Close
lar
Dong Shick
Insert Peg
Meat off Giill
Open Drower
Place Cups
Place Wine
Ruttons
Push
Imuge-Bc BC (CNN)
21
19
1.3
11:72
0.0
0.0
0.0
0.0
0.0
40
0.0
0.0
Image BC (WT)
21
59
13
12.19
0.0
0.0
0.0
0.0
0.9
0.0
0.0
0.0
C2F-ARM-B BC
20
20.1
10.72
24.0
26.0
20.0
20.0
8.0
72.0
HiveFormer
8.47
76.0
100.0
800
PolarNer
92.0
100.0
84.0
400
[B
92.0
95.0
RYT
4.92
52.0:2.5
99.2
11:2330
88.0=25
71,2166
40:2.5
9100552
100.0:000
Ditfuset Actor
81.3
2.67
96.0223
100.0...0
63.624..1
1.
89.644.1
24.017.6
936-4.8
98.422.0
RVT=2
[S]
81.4
2.75
1000+1.00
990=17
400++00
99001.7 1.7
74.0+118
54.D44.5
95.0-33
10M0=100
Bridge VLA vilo heal
31.4
10.06
49.312.3
6
0.010.00
813
74.7.100.1
131233
32.01144
54.716..
HridgeYLA pos
562
5.97
9000000
58.756.1
26,7622
95000.0
97.3822
14.254.6
81.3=333
86.742.
BridgoVVLA
88.2
2,03
1000+0000
1000-0.0
100.0-000
100,0-0.0
5844100 10.0
88.0-28 28
98.442.2
Models
Capboard
Pur
Drower
Put
is
Safe
Screw
Bulb
Block
Side
Shape
Sont
Blocks
Stcck
Stack
Cups
Sweep to
Dosipin
Tans
Top
Image (CC (CNN) 21
59
0.0
8.0
4.0
0.0
0.0
0.0
0.0
0.0
0.0
8,0
InaageBCC (VTT)
21
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
16.0
C2F-ARM-BB
20
12.0
8.0
16.0
8.0
0.0
0.0
68.0
HiveFormer
8.0
8.0
28.0
PolarNer
12.0
44.0
4.0
52.0
51,21447
26.41332
$2.0.000
12.0
92.0
RVT
49.8-12
88.0+5
91 31:30:00
48.0=5.7
360-2.5
28.8+3.9
72:0000
91.614.1
Diffuset Actor
85.6...1
96.01136
97.612200
82.1120
97.6132
64.0.4.44
68.3133
47.218.5
84.0...4
99.2116
RVT-2 [[S]
6..0 4.5
96,0.100
96.01.88
88.0
92.128
35.0 7.1
80.01288
69.015.9
100.0 0.0
99.011.7
Bridge LL wio heat
5.5+22
D0:00
58.7 22.7
2.742.
640:00
4.0s400
0.0:00
0.200
$2.0=10
10.
BridgeV1 w pos
10.7-2.3
78.7+2.3
97 34-46
16,0+40
72.+00
213--8.
17,342.3
40+40.0
53.3-2.
840+00
BridgeVLA
73.6-.6
99.2118
99.21..8
87.216.6
96.0288
60.11.
76.818.7
$1.6336
872
928133
Table I: Results on RLBench. The "Avg. Rank" column reports the average rank of each method across all 18 tasks, where lower values indicate better overall performance. "BridgeVLA w/o heat" refers to the ablated version that directly predicts actions without using intermediate heatmaps. "BridgeVLA w pos" refers the ablated version that incorporates position features into the image features. Bridge VLA achieves the best performance in 10 out of the 18 tasks.
into bins, we also apply cross-entropy loss in Tron to supervise rotation prediction. For gripper action and collision avoidance, we use the binary cross-entropy loss in /griiper and T-collision as supervision. To enhance geometric robustness, random rigid-body transformations are applied jointly to the point cloud and the ground-truth action during training. Additional training details can be found in Appendix
Experiments
In this section, we perform extensive experiments in both simulation and the real world to evaluate the proposed method. Through the experiments, we aim answer five questions:
QI: How effectively does Bridge learn 3D robot manipulation compared to state-of-the-art methods when sufficient data available?
Q2: Does Bridge learn more efficiently than existing state-of siie--of-the the-aart methods when data is limited (e.g., 3 trajectories per task)?
Q3: How robust is Bridge VLA and lighting)?
VL. in handling visual disturbances (e.g., distractors, background,
Q4: How well does BriA generalize novel object-skill combinations and objects from previously unseen categories?
Q5: Are our architectural designs (c.g., predicting heatmaps before outputting actions) truly useful when constructing 3D VLA?
4.1 Simulation Experiments
4.1.1
Experiments on RLBench
Setup. RLBench 19 implements tasks CoppeliaSim (38 using a Franka Panda robot mounted with a parallel-jaw gripper. The observation contains four RGB-D images captured from four calibrated cameras positioned the front, left shoulder, right should, and wrist. Following previous 25 15), we perform experiments on 18 tasks from RLBench. These tasks span I)
works 39 13 14

---

non son-prehensile manipulation (e.g., slide block to target), 2) pick-and aae--and-paace -lace (e.g., stack cups), and 3) high- nigh-precision insertion (e.g. insert peg). Each task provided with 100 expert demonstrations, And each demonstration is paired with language instruction and multiple keyframes. Models are evaluated via binary success rates over 25 trials per task, with a maximum of 25 action steps per trial.
Baselines. We compare BridgeVLA with multiple baselines. (1) Image-BC (CNN) and Image-BC (ViT) [21] are two 2D baseline methods which predict the actions directly from 2D images using CNN and ViT the backbone, respectively. (2) C2F-ARM-BS (20) predicts the next keyframe action in the voxel space with a coarse-to-fine strategy. (3) PerAct [39] also operates in the voxel space and predicts the action with a perciever transformer [8]. (4) HiveFormer incorporates
historical information using a unified multi-modal transformer architecture. (5) PolarNet employs PointNext (36 encode the 3D scene and predicts both heatmaps and offiets for all points to estimate translational actions. (6) Act3D 13 predicts the next keyframe action by selecting the point with the highest score from set of randomly sampled points in the workspace. (7) 3D Diffuser Actor [25] generates 3D trajectories via a diffusion process conditioned on 3D observation
and language instructions. (8) RVT [4 uses multi-view transformer to aggregate information
from multiple orthographic views of the point cloud observation. (9) And RVT-2 [15 the current
state-of-the-art method, further improves the precision of its prior via a coarse-to to-rie-- strategy.
Results. In total, we evaluate Bridge VLA five times minimize statistical bias. The results shown in Table 1 BridgeVLA outperforms all the comparing baseline methods, achieving an average success rate of 88.2% and average rank of 1.9 across all the 18 tasks, establishing new state of the art in this benchmark. These results address QI, demonstrating the effectiveness of Bridge in learning complex 3D manipulation tasks. We highlight that BridgeVLA outperforms the best baseline method by large margin in Insert Peg (88.0% vs 40.0%) and Sort Shape (60.8% vs 35.0%). These two tasks demand highly precise alignment between the peg and hole and the block and sorter, respectively. The high success rates of our method showcase its strong capabilities of learning precise manipulation which is highly desirable in many industrial applications. Among the 18 tasks, BridgeVLA performs the worst in Place Cups, despite surpassing all the comparing baseline methods. We hypothesize this is because the target keypoints are often occluded in all orthographic projection views, which makes learning and prediction more challenging. In the future, we plan to explore dynamically selecting the projection views for rendering to avoid this problem.
4.1.2 Experiments on COLOSSEUM & GemBench
To further evaluate the generalization capabilities of BridgeVLA, we conduct experiments on the COLOSSEUM benchmark [35] and GemBench [12]. These two benchmarks extend RLBench. The COLOSSEUM benchmark evaluates models in environments with 12 axes of perturbations, which were not seen during training. These perturbations include variations in object texture, color, size, background, lighting, distractors, and camera poses. As such, this benchmark is used to assess Q3.
GemBench is a hierarchical generalization benchmark. Its training set consists of 16 tasks (31 variations) covering seven core action primitives: press, pick, push, screw, close, open, and stack/put. The test set includes 44 tasks (92 variations), categorizee into four increasingly challenging settings. These settings incorporate novel object-skill combinations and new object categories, making suitable for evaluating Q4.
Bridge VLA outperforms all existing state-of-the-art 3D manipulation methods both benchmarks, addressing Q3 and Q4. Due to space limitations, the details the environment setup, baselines, and analysis can be found in Appendix B.I and Appendix B.2
4.2 Real-Robot Experiments
Setup. In this section, we perform real-robot experiments to validate the effectiveness of Bridge VLA in the real world. Our real-robot setup includes a Franka Research robot arm mounted with a parallel-jaw gripper (Fig. 33. A static ZED 2i depth camera is used to provide the colored point cloud observation. total, we evaluate on 13 tasks (see Tab. 12 for a full list of tasks). These tasks ranges from simple pick-and-place to complex long-horizon tasks, requiring the robot to open drawer and put items into the drawer. Each task contains 3-9 keyframes (see Fig. and for visualization). For each task, we collect 10 expert trajectories for training.
7

---

Mathod
Pui the sola
the bottom sheff
Put the graffe the lnne drawer
Place the red block in the bla plate
Press Sanilizee
Put the RedBull cun the sholf
Put the RedBull cus in the button shell
Pur the cole in the shelf
SponoVV-AA100
1/10
1/10
5/0
5/1
3/0
1/10
2/10
0710
010
0/10
0/10
53
2/10
3/10
3/0
1/30
RVT- L13
10/10
8/10
8/10
10/10
9010
I0V10
1020
VLA
$010
9/10
10710
10710
10710
IW10
Method
Place the orange block
the groce plnne
Place the red block in the pepplc plnne
Place the yellow baoc
thr groz plats
Pur the zotua
upper deower
Put the sebra
lnwer swer
the apper Inswer
Put die wolf
Avergee
SpuialVLAASo 53
6/10
5/10
0/10
2/10
28.5
1/10
1/10
W10
W10
0/10
030
3:166
010
0/10
V/10
IV10
0/0
0/10
3,8%
AGT L3
2/10
3/10
4/10
1/10
2010
1/10
12 99
RVB2 (13
10/10
9/10
W10
7/10
R10
9/0
9078
BasgeeLA PLA
10/10
1071
[0V10
910
1010
9/0
96.945
Table 2: Per-task Success Rate in the Basic Setting. Except for SpatialVLAA(OO, which was trained with 50 trajectories, all other methods were trained with 10 trajectories. BridgeVLA outperforms all baseline methods, achieving an almost perfect success rate of 96.9%.
In total, we design 7 different settings comprehensively evaluate our model's performance. (1) Basic: The model is evaluated in environments that are similar to the training data. (2) Distractor: Distractor objects that are visually similar to least one target object are added to the scene. (3)
Lighting: The model is tested in a visually distinct lighting condition in which the lights are turned off. (4) Background: Three different tablecloths are used to change the background. (5) Height: All objects for manipulation are placed on drawer that is 9.5cm high. (6) Combination: We combine objects and skills that are not paired together in the training datasets. That is, while the objects (e.g., red block and green plate) and skill (e.g.. place A in B) are seen during training, the instruction that pairs them together is novel (e.g.. place the red block in the green plate). In total, we evaluate 13 novel object-skill combinations (Fig. [1 and 12 (7) Category: To test whether Bridge is able to transfer the broad knowledge from pre- training to downstream policy learning. we evaluate on manipulating objects from categories that are unseen in the robot training data. In total, we test 7 novel objects (Fig. [.. Distractor, Lighting, Background, and Height aim to evaluate the robustness against visual disturbances, while Combination and Category evaluate the generalization capabilities for unseen instructions.
To demonstrate BridgeVLA's advantages over existing manipulation policy, we compare it with four types of representative methods:
1) SpatialVIA [37]: A state-of-the-art 3D VLA model that incorporates 3D information through Ego3D positional encoding and leverages Adaptive Action Grids to accelerate inference.
2) To 5]: A state-of-the-art 2D VLA model pretrained on a large-scale cross-embodiment dataset. It adopts vision-language model (VLM) backbone and employs a flow matching action expert to generate final actions.
3) ACT :1 A state-of-the-are 2D nLn-VLA model using a Conditional Variational Autoencoder (CVAB) to model action distributions. Though effective for fine-grained manipulation, ACT does not support language conditioning, so we train a separate single-task model for each task, which should theoretically perform better than a multi-task version.
4) RVT-2 15: A -ate-of-the-art 3D non-VL. VLA model performing the best in our simulation experi- ments. (See Sec. 4.1
Results. We first compare Bridge VLA with these baselines the basic setting. For every task, we evaluated every baseline over 10 trials to ensure statistical robustness. For fair comparison, we photographed each test scene and manually aligned the scenes across all methods. Results are provided in Tab. 2 As we can see, most methods completely fails when given only 10 trajectories per task except two 3D related methods: RVT-2 and BridgeVLA. Notably, although SpatialVLA also utilises 3D information, its data efficiency is still very low. Even when the data are increased to 50 trajectories per task, its success rate is still much lower than Bridge 3D information is not enough for constructing 3D VLA model and a carefully designed network architecture is still very important. To assess the data efficiency of BridgeVLA, we also train the model with only 3 trajectories per task. Remarkably, despite the limited data, BridgeVLA achieves a success rate of 95.4% in Basic, matching the performance achieved with 10 trajectories per task. This result underscores the data efficiency of the proposed method, directly addressing Q2. Detailed
which indicates only adding
8

---

Real Robot Setup
Basic Setting
Generalization Settings
Background
Lighting
Distractor
Franka Research 3
ZED
Combination
Height
Category
RVT-2
BridgeVLA w/o Pre-train
BridgeVLA (Ours)
1.0
Success Rate
0.8
0.6
0.4
0.2
0.0
Basic
Distractor
Lighting
Background
Height
Combination
Category
Figure 3: Real-Robot Experiments and Results. We use a Franka Research 3 robot arm and a ZED 2i camera capture point clouds of the scene. To evaluate the model's performance, design different settings including one basic setting and six generalization settings. Experimental results show that Bridge VLA outperforms the state-of-the-art baseline method RVT-2 [15 by average of 32%.
per-task results are provided in Appendix C.5 and observations about the baselines are detailed in Appendix CC.2
Considering that only RVT-2 and BridgeVLA have good performance in basic setting, we only further evaluate these two methods on other generalization settings. The average success rates are shown in Fig. 3 BridgeVLA outperforms RVT-2 in all the seven settings. As struggles in both visual generalization settings and semantic generalization settings, while BridgeVLA performs much better, especially in Lighting and Combination. These results addresses Q3 and Q4, indicating that BridgeVLA is able to handle visual disturbance and novel instructions robustly.
see, RVT-2
Although our method outperforms baseline methods in the Category setting, its absolute success rate is not high. A common failure mode is that the robot often ignores the target object and moves directly to the destination dur- ing aad-ane manipulation. We believe this relatively low performance is not due to Bridge VLA forgetting the knowledge gained from traininiing it still predicts heatmaps accurately when provided with samples from the pre-training dataset after fine-tuning (see Fig. 4 and Appendix C.4).. Instead, we hypothesize that the reduced performance stems from two factors: 1) The images in the pre-training dataset are mostly captured from third- perd-pprsoon views, which differ significantly from the projection images in our robot data; The pre-training task focuses solely on object localization, whereas manipu- lation involves predicting keypoints that do not correspond to an object. To address these issues, we plan to expand both the scale and diversity of the pre-training dataset and explore more expres- sive decoddng methods better leverage the preserved pre-training knowledge.
Sneakers
Apples
Pandas
Input Image
Prediction
Ground- Truth
Figure Prediction Pre-training Data after Fine-tuning. To simulate the multi- view inputs during fine-tuning, we repeat each pre-training im- age three times and feed them into the fine-tuned model to generate heatmaps. Note that these sam- ples are not cherry-picked. Additional samples can be found in Appendix C.4

---

4.3
Ablation Studies
To prove the effectiveness of our model design and provide insights for the community, we conduct three ablation studies:
Whether need to predict heatmaps before predicting actions. Our approach avoids direct action prediction by first generating 2D heatmaps using a convex upsampling module. Target positions are then computed by projecting 3D workspace points onto the heatmaps and selecting the point with the highest mean probability. For ablation, we replaced the convex upsampling module (309M parameters) with similarly sized Transformer decoder (303M) directly predict target positions, supervised by MSE loss. All other modules were kept the same as before. We performed a hyperparameter grid search and evaluated the model on RLBench. Results are shown in the Tab. [ Replacing heatmap prediction with direct position regression reduced the average success rate from 88.2% to 31.4%, confirming the effectiveness of our heatmap-basee design. The ablated model was also harder to train and more sensitive to hyperparameters requuruineerr-rruuiinngggg batch size of 192 and careful learning rate tuning-war whhle our original model trains reliably even with a batch size of 64. We see three main reasons for this outcome: (1) Heatmaps offer denser supervision than 3D position vectors, enabling more effective learning. (2) Projecting 3D points onto heatmaps introduces helpful spatial priors, easing the learning process, (3) The 2D heatmaps share the same spatial structure as the input images, enhancing alignment and improving performance.
Whether we need to remove the 3D position input to the VLM backbone. Unlike typical 3D VLA models like SpatialVLA, we deliberately avoid using per-pixel 3D position inputs and rely solely on RGB images. This design preserves alignment between the input feature spaces of fine-tuning and VLM pretraining, which we find crucial for effective vision- aarion-lnngagg-acttonn test this, we added 3D convolutional module to encode per-pixe) 3D positions, fused them with 21) features, and fed the result into the backbone. Although this adds richer spatial cues, alters the image feature distribution seen during pretraining, leading to performance drop from 88.2% to 56.2% on RLBench Detailed results are shown in the Tab. [
(VLA) modeling. To
Whether we need to do 2D heatmap pre-training to the VLM backbone. The original VLM backbone can not predict heatmaps, while our downstream policy learning requires such ability. To bridge the gap, we do 2D heatmap pre-training to the VLM backbone. To verify its effectiveness, we ablate this pre-training and evaluate model's performance in the real world, the results are shown in Fig. 3] As we can see, Bridge VLA w/o Pre-train is not able to generalize well in both language rellttd generalization settings and can not even beat RVT-2, while Bridge achieves the best performance across the two generalization settings especially in Combination, highlighting its ability to understand language semantics. We hypothesize that the 2D-heatmap pre-training equips Bridge with the ability to connect the semantics in language instructions with image observations in the heatmap space. All the above experiment results address Q5 and highlight the effectiveness of our architectural designs.
5 Conclusions & Future Work
This paper has introduced BridgeVLA, novel and efficient 3D vision- sssion-lnguaag--cttio (VLA) model built on top of a pre-trained vision-language model (VLM) [... Keys to our method are that (I) it converts 3D inputs to 2D images to align with the 2D image inputs of the pre-trained VLM; (2) it aligns the input observation and the output action to a unified 2D image space via 2D heatmap prediction; (3) it adopts scalable pre-training method to equip the VLM with the capability to predict heatmaps before fine-tuning on action prediction. Extensive experiments show that the proposed method is able to learn 3D manipulation efficiently and effectively in both simulation and the real world. In the future, we plan to explore pre-training on more diverse tasks, including semantic segmentation and keypoint detection. We also want incorporate more expressive action-decoding methods (e.g., diffusion (00) into the framework to continue improving the policy performance.
6 Acknowledgments
This work was jointly supported by National Natural Science Foundation of China (62322607, 62236010 and 62276261), Beijing Natural Science Foundation (1.252033), FiveAges Grant, and Youth Innovation Promotion Association of Chinese Academy of Sciences under Grant 2021 128.
10

---

References
[1] AI@ Meta. Llama 3 model card. 2024, URL blob/ #lob/main/MODEL_Coo md
[2] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
[3] Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, al. Paligemma:: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726, 2024.
[4] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi. Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. m0: A Aasionnlanguage-ne model for general robot control, 2024. URL hups://urxiv. htt//bxx/211.224160
flow
[5] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. \mp: A mision-language model for general robot control. arXiv preprint 4rX1v:2410.24164, 2024.
flow
[6] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022.
[7] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, al. Rt-2: Vision-language- action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818 2023.
[8] Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-laggugee-ctioon model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.0613 2024.
[9] Shizhe Chen, Ricardo Garcia Pinel, Cordelia Schmid, and Ivan Laptev. Polarnet: 3d point clouds for language-guide robotic manipulation. In Conference Robot Learning, pages 1761-1781. PMLR, 2023.
[10] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2024.
[11] David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the barrier to entry of complex robotic software: moveit! case study. arXiv preprint arXiv:1404.3788 2014.
[12] Ricardo Garcia, Shizhe Chen, and Cordelia Schmid. Towards generalizable vision-language robotic manipulation: A benchmark and Ilm-guided 3d policy. arXiv preprint arXiv:2410.01345, 2024.
[13] Theophile Gervet, Zhou Xian, Nikolaos Gkanatsios, and Katerina Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. In Conference on Robot Learning, pages 0949-3965. PMLR, 2023.
[14] Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox. Ryt: Robotic view transformer for 3d object manipulation. In Conference Robot Learning, pages 694-710. PMLR, 2023.
[15] Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. Rvt-2: Learning precise manipulation from few demonstrations, In RSS 2024 Workshop: Data Generation for Robotics, 2024.
[16) Pierre-Louis Guhur, Shizhe Chen, Ricardo Garcia Pinel, Makarand Tapaswi, Ivan Laptev, and Cordelia Schmid. Instruction-driven history-aware policies for robotic manipulations. In Conference Robot Learning, pages 175-187. PMLR, 2023.

---

[17] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. x0.5: a vision- language-action model with open-world generalization. arXiv preprint arXiv:2504. 16054, 2025.
[18] Andrew Jaegle, Sebastian Borgeaud, Lean-Baptise Alayrac, Carl Doersch, Catalin lonescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. In International Conference on Learning Representations.
[19] Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Roboties and Automation Letters, 5(2):
2020.
[20] Stephen James, Kentaro Wada, Tristan Laidlow, and Andrew J Davison. Coarse-to-fine q- attention: Efficient learning for visual robotic manipulation via discretisation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13739-13744, 2022.
[21] Eric Jang, Alex Irpan. Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning, pages 991-1002. PMLR, 2022.
[22] Yueru Jia, Jiaming Liu, Sixiang Chen, Chenyang Gu, Zhilue Wang. Longzan Luo, Lily Lee, Pengwei Wang, Zhongyuan Wang, Renrui Zhang, et al. Lift3d foundation policy: Lift- ing 2d large-scale pretrained models for robust 3d robotic manipulation. arXiv preprint arXiv:2411.18622 2024.
[23] Edward Johns. Coarse-to-fine imitation learning: Robot manipulation from single demon- stration. In 2021 IEEE international conference on robotics and automation (ICRA), pages 4613 4619. IEEE, 2021.
[24] Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. In Forty-first International Conference on Muchine Learning, 2024.
[25] Tsung-We Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3d diffuser actor: Policy diffusion with 3d scene representations. In 8th Annual Conference on Robot Learning, 2024.
[26] Moo Jin Kim, Karl Pertsch, Siddharth Karameheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, et al. Openvla: An open-source Assion-language-actionn model. In 8th Annual Conference on Robot Learning.
[27] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02664 002443000.43.4.. 2023.
[28] James Kuffner and Steven M Valle. Rrt-connect: An efficient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium conference. IEEE international conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065), volume 2, pages 995- 1001. IEEE, 2000.
[29] Chengmeng Li, Junjie Wen, Yan Peng, Yaxin Peng, Feifci Feng, and Yichen Zhu. Pointvla: Injecting the 3d world into ussionlanguaag-actinn models. arXiv preprint arXiv:2503.07511 2025.
[30] Peiyan I.,, Hongtao Wu, Yan Huang, Chilam Cheang, Liang Wang, and Tao Kong. Gr-mg: Leveraging partially-anootate data via multi-modal goal-conditioned policy. IEEE Robotics and Automation Letters, 2025.
[31] Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wa, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision- -lnguan foundation models as effective robot imitators. arXiv preprint arXiv:2311.01378, 2023.
12

---

[32] Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling ooen-vocyy object detection, Advances Neural Information Processing Systems, 36:72983- 2023.
[33] Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203. /2601, 2022.
[34] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07799. 2023.
[35] Wilbert Pumacay, Ishika Singh, Jiafei Duan, Ranjay Krishna, Jesse Thomason, and Dieter Fox. The colosseum: A benchmark for evaluating generalization for robotic manipulation. arXiv preprint arXiv:2402.08119/ 2024.
[36] Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in neural information processing systems, 35:23192-23204, 2022.
[37] Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang. Yian Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual-language-actioo action model. arXiv preprint arXiv:2501. 15830, 2025.
[38] Eric Rohmer, Surya PN Singh, and Mare Freese. V-rep: A versatile and scalable robot simulation framework. In 2013 IEEE/RSJ international conference on intelligent robots and systems, pages 1321-1326. IEEE, 2013.
[39] Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pages 785-799. PMLR, 2023.
[40] loan A. Şucan, Mark Moll, and Lydia E. Kavraki. The Open Motion Planning Library. IEEE Robotics & Automation Magazine, 19(4):72 82, December 2012. doi: 10.1 00.1109/RRA.001 2205651.
kavrakiia......a....a....... org
[41] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08293 2024.
[42] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Purt 11 16, pages 402-419. Springer, 2020.
[43] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191 2024.
[44] Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu. Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation. In International Conference on Learning Representations, 2024.
[45] Xiaoyang Wu, Li Jiang, Peng-Shual Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Proceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pages 4840 4851, 2024.
[46] Tete Xiao, Ilija Radosavovie, Trevor Darrell, and Jitendra Malik, Masked visual pre-training for motor control. arXiv preprint arXiv:2203.06/73 2022.
[47] Rujia Yang, Geng Chen, Chuan Wen, and Yang Gao. Fp3: A 3d foundation policy for robotic manipulation. arXiv preprint arXiv:25033089500 2025.
[48] Wentao Yuan, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. M212: Multi-task masked transformer for object-eentric pick and place. arXiv preprint
2023.
13

---

[49] Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. Robopoint: A vision-language model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406. 10721, 2024.
[50] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. Proceedings of the IEEE/CVF international conference on computer vision, pages 1975--1986. 2023.
[51] Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprini arXiv:2304. 13705, 2023.
[52] Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d iision-lagguage-acttin generative world model. arXiv preprint arXiv:2403.09631, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to Abstract and Sec. 1
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss our limitations Sec. and Sec. S
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification: All results in this paper are supported by experiments and we do not include any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: have showed the main information our method and experiment settings in the main paper. Meanwhile, we also display details in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We will opensource the code, data and checkpoints upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https
/ "pipsscc/puullc//uuu /otee///uuddde/eooSuuubbmiibii ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have specified these details in Sec.
Sec.
Sec.
Sec.
B
and Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report both the success rate and variance Tab. Tab, 6 and Tab.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
17

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide these information in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We did not identify any aspects that violate the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work introduces a novel 3D VLA framework aimed at advancing the AI and robotics community. We do not identify any direct societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not find such risks in this project.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: For the code, data and other assets we use in this paper, we have cited original paper and respected their terms.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

Appendix
Table Training hyperparameters for Bridge
Pretrain
RLBench Finetune
Colosseum Finetune
Real-robot Finetune
learning rate
5e-5
8e-5
8c-5 AdamW W 192
2e-5 AdamW W 192
optimizer
AdamW
AdamW
batch size
384
192
warmup steps
400
A Training & Inference Details
Detailed training configurations summarized in Tab. 3 Throughout both pre-training and fine- tuning, we keep the SigLIP vision encoder and language token embeddings frozen.
Computational Resources:
1. Pre-training: 8 NVIDIA A100 GPUs for 3,800 steps (fa2 hours)
2. RLBench fine-tuning: 48 NVIDIA H100 GPUs for 83,000 steps (#:20 hours)
3. COLOSSEUM fine-tuning: 48 NVIDIA H100 GPUs for 83,000 steps (#:20 hours)
4. GemBench fine-tuning: 40 NVIDIA A A100 GPUs for 50 epochs (Rx2. hours)
5. Real-world fine-tuning: 8 NVIDIA A A100 GPUs for 300 epochs (s 1.5 hours)
For inference, we run Bridge on a machine equipped with an NVIDIA RTX 4090 GPU. To evaluate its inference speed, we conducted 100 trials. From point cloud input to action output, the average end-to-end inference time is 0.21 seconds.
B Simulation Experiments
B.1 Experiments on COLOSSEUM
Setup. The COLOSSEUM benchmark is an extension to the RLBench benchmark. The model is trained the data from the original RLBench benchmark but evaluated in environments spanning 12 axes of perturbations. These perturbations, which are unseen during training, encompass changes in object texture, color, and size, backgrounds, lighting, distractors and camera poses. In total, the COLOSSEUM creates 20.371 unique task perturbations instances to comprehensively evaluate the generalization capabilities of the model. Specifically, our evaluation includes three steps: 1) train the model with the original RLBench data without perturbations (100 trajectories per task) on 20 tasks, 2) evaluate each task over 25 trials per perturbation, 3) compute the average success evaluated tasks for every perturbation. Besides the 12 types of perturbations, we also evaluate on basic variations from the original RLBench (denoted RLBench Tab, 4), and more challenging setting which combines all the 12 types of perturbations (denoted as All Perturbations in Tab. 4
of all
Baselines. We compare Bridge VLA with five baseline methods. R3M-MLP and MVP-MLP are two 2D methods that utilize pre-trained visual encoders process observation images and an MLP for action prediction. Specifically, R3M-MLP uses R3M 33 that is pre-trained on large-scale egocentric human videos; MVP-MLPP uses MVP [46] that is pre-trained on millions of in-the-wild data. Both visual encoders show strong adaptability on various robotics tasks in both simulation and the real world. We also compare with three 3D methods introduced in Sec. ie.. PerAct [39]. RVT [4], and RVT-2 [15).
22

---

Models
Avg. (%)
Arg.
Perturbations
MO-COLLOR
RO-OOLOR
MO-TENTURE
RO-TENTURE
MO-SIZE
R3M HAM-MLMSS
0.8
571
0.6
0.4
0.0
0.0
13
MVP-MEP
08
1.2
0.0
0.4
4.44
ParAcT
27.9
3.71
7.2
24.0
29.2
28.3
17.21
35.5
RVT
35.4
3.28
6.4
26.0
31.3
44.8
41.1
35.3
RV
56.7
192
15.6 0.
53.0 + 0.9
$46 #06
59.7
56.7 + 144
60.9
BridgeVLA (Ours)
64.0
1.07
18.7
68.5
I.I
638 :
63.5
1.5
68.4 :: 3.3
69.3 1.0
Models
RO-SI2D
Light Coker
Tuble Coke
Tabll Texture
Distractce
Dackground Texture
RLBench
Camera Pose
R3M MLM
0.0
1.0
1.4
0.2
1.6
1.2
2.0
0.8
MVP MLP.MLP.GR dh
0.0
1.6
1.6
1,0
3.8
2.2
2.0
2.6
PerAat 13
29.3
29.1
30.4
23.2
27.1
33.5
39.4
36.3
RVT [4]
40.5
54.0
10.0
45.2
18.5
46.4
53.4
42.2
RV
53.4+155
58.
1.1
62.6+ 0.9
56.6 +
68.8 + 0.5
68.7 +
68.6 +
64.4 0.5
Bridge (Ours)
61.7 0.8
69.7
1.2
15/7100
71.3
71.3.677
51.8
74.8
73.1
73,8
Table 4: Results on the COLOSSEUM Benchmark. The table shows the success rates across 14 generalization settings. The "Avg. Rank" column reports the average rank of each method across all perturbations, where lower values indicate hetter overall performance. Compared to the state-of-the- art baseline, BridgeVLA improves the average success rate by 7.3%.
Results. Results shown in Tab. 4 We use the results of R3M-MLP 133
MVP-MLP [46). For RVT-2 [IS and
RVT [14), and PerAct [39] from the original COLOSSEUM paper (35.
Bri,geVLA, we perform own training and evaluation process. We performed three test repetitions and report the average success rate and variance of BridgeVLA and RVT-2 for each task under different perturbations in Tnd and Tab respectively. BridgeVLA outperforms all the comparing baseline methods in terms of average success rate, significantly outperforming the best baseline method by 7.3%. Among all the 14 evaluated perturbations, method ranks the best among all methods in 13 of them. These results address Q3, showcasing that Bridge VLA possesses strong robustness against visual perturbation.
B.2 Experiments on GemBench
Setup. GemBench [12 is hierarchical generalization benchmark built the RLBench simula- tor [9] Its training set contains 16 tasks (31 variations) covering seven core action primitives- presss,, pick, push, serew, close, open, and stack/put. The test set consists of 44 tasks (92 variations), categorized into four increasingly challenging settings:
L1 (Novel Placements): LI consists of the original 16 tasks (31 variations). The object placements are randomized within the workspace. In addition, chromatic distractors are introduced to test the ability to handle additional visual complexity.
L2 (Novel Rigid Objects): L2 involves 15 unseen tasks (28 variations) that require interaction with novel rigid objects using learned primitives. The generalization capabilities are evaluated across two categories: novel object-color compositions and novel object shapes.
L3 (Novel Articulated Objects): L3 consists of 18 unseen tasks (21 variations) that involve interact- ing with articulated objects. It evaluates the generalization capabilities across three categories: novel action-part compositions, novel object instances, and novel object categories.
L4 (Novel Long- Long-Horizon Tasks): L4 includes 6 complex long-horizon tasks (12 variations) that require combining multiple actions to finish whole task.
Baselines. In total, we compare with six baseline methods. 3D-LOTUS
processes point cloud point cloud transformer architecture 45 It showcases notable
inputs through a multi-tasking capabilities and high training efficiency. Its enhanced variant, 3D-LOTUS++ [12], integrates the generalization capabilities of large-scale models into 3D-LOTUS with modular architecture consisting of three components: (1) LLLM--asdd task planning M, (2) VLM--bssed object grounding [32 (2), and (3) motion control inherited from 3D-LOTUS. We also compare with four methods introduced in Sec. 4.1.I i.e., Hiveformer [16], PolarNet [9) 3D Diffuser Actor (25)
RVT-2 [5
Results. Overall results are shown in Tab. 5 and per-task success rates on the four settings of 10 TT The results of baseline methods are sourced from [12].
GemBench are shown Tab. g In total, we evaluate on 5 random seeds to reduce statistical variance. And for every seed,
23

---

Method
Average
L1
L2
L3
L4
Hiveformer [6
30.4
60.3 1.5
26.1 + 1.4
35.1 1: 1.7
0.0 0.0
PolarNet 9
38.4
77.7 77.7-0.9
37.1 - 1.4
38.5 1 1.7
0.1
3D Diffuser Actor
91.9
37.0 2.2
0.0
RVT-2 15
44.0
8.81
51.0
36,0 + 2.2
0.0 0.0
3D-LOTUS 12
45.7
= 1.4
49,9 2.2
38.1 1.1
0.3 0.3
3D-LOTUS-
48.0
68.7 = 0.6
64.5 0.9
41.5 +
1.8
17,4 0.4
BridgeVLA (Ours)
50.0
91.1
65.0 1.3
43.8 +
1.2
0.0 0.0
Table Results on GemBench. We show the average success rates on the four evaluation settings of GemBench. BridgeVLA establishes new state of the art this benchmark, achieving average success rate of 50.0%.
20 trials per task variation. BridgeVLA consistently outperforms all the comparing baseline methods in terms of average success rate across the four evaluation settings. Notably, BridgeVL. achieves tate-of-the-ar results in both the L.2 and L3 settings, demonstrating strong generalization capabilities, addressing Q4. However, similar to most baseline approaches, Bridge exhibits limited performance in the L44 setting, where each task comprises multiple sub-tasks. In the future, we plan to explore leveraging large language models (L.LMs) for long-horizon task decomposition and further improve the performance in such setting.
B.3 Key frame Selection
For all the simulation and real-robot experiments, we adopt the same key frame selection strategy as PerAct [39]. A time step is labeled as key frame if (i) the robot is stationary, (ii) the gripper state changes, or (iii) the step is the final state of the episode. The robot is considered stationary when the absolute velocities of all joints fall below 0.1 rad/s.
B.4 Data
Following 39 15 we select tasks from RLBench 19 to evaluate the performance of our method complex manipulation tasks. These tasks are visualized in Fig. 5
To assess the generalization capability of VLi, also evaluate the COLOSSEUM benchmark [35] and GemBench [12]. The COLOSSEUM benchmark includes 20 basic tasks and 12 types of perturbations. These perturbations, which are unseen during training, encompass changes in object texture, color, and size, backgrounds, lighting, distractors and camera poses. The benchmark evaluates on all the 12 types of perturbations, setting with basic variations from the original RLBench, and more challenging setting which combines all the 12 types of perturbations. We visualize all perturbations except the one from the original RLBench in Fig.
For GemBench, the training set includes 16 tasks (31 variations) spanning seven fundamental action primitives (press, pick, push, screw, close, open, stack/put). The test set includes 44 tasks (92 variations) organized into four increasingly challenging settings. Unlike RlBBench and COLOSSEUM, where demo augmentation used, we train BridgeVLa VLA using only keyframes from each trajectory without performing any demo augmentation in GemBench.
C Real-Robot Experiments
C.1 Experiment Setup
Fig. illlsttraees our real-robot setting. The platform comprises a 7-DoF Franka Research 3 manipu- lator with a parallel-jaw gripper and a ZED stereo camera mounted on tripod for capturing point clouds of the workspace. We collect expert trajectories with kinestheic teaching approach. We first move the manipulator to keypoints of an expert trajectory and then play back the keypoints to record the observation and action at each keypoint.
24

---

C.2
Basic Setting
This setting provides a scene similar to the training dataset, where only the object layouts are modified. To highlight BridgeVLA's advantages over existing manipulation policies, we compare it with four representative methods in this setting. The behaviors of these baselines are as follows:
Sp 37 In the experimental setup, we initially trained SpatialVL. using only 10 trajecto- ries per task. However, it failed nearly all tasks, often struggling to move toward the correct target object. To improve performance, we augmented the dataset with an additional 40 trajectories per task. While this improved performance, it still lagged significantly behind Bridge ALA---atiiulllrtt on more challenging tasks, such as "Put the giraffe in the lower drawer." These findings suggest that BridgeVLA provides a more effective and data-efficient solution for 3D VLA.
To [3] Similarly, To fails with only 10 trajectories per task, likely due to overfitting it performs well on the training set but often fails during online testing. Common failure modes include missing or failing to grasp the target and prematurely opening the gripper before reaching the goal. Notably, both BL. and To share the same PaliGemma backbone and are trained end-to-end. This highlights a key contribution of our work: while VLAs like To perform well with large-scale data, they struggle in low-data regimes- even simpler tasks, such as "Press sanitizer." In contrast, BridgeVL. VLA achieves near-perfect success and generalizes robustly across diverse settings.
ACT [51] ACT also underperforms compared to BridgeVLA, It demonstrates limited spatial generalization, performing well only in areas densely covered during training, but often failing when the target is near the workspace boundaries. This behavior is consistent with its design: ACT models actions using Gaussian prior, which assigns low probability to peripheral regions, limiting its spatial generalization capabilities.
RVT-2 [15]: RVT-2 performs the best among all the baselines. It can successfully solve most tasks, but it is not as robust as Bri, For instance, it sometimes fails to pick up the block precisely or place the object accurately, leading to task failure. Meanwhile, by utilizing the capabilities of VLM, Bridge VLA's advantages are further amplified in generalization settings, as detailed in Sec. 4.2
C.3 Generalization Settings
We evaluate on total of six generalization settings: Distractor, Lighting, Background, Height, Combination, and Category. For Distractor, Lighting, Background, and Height, we visualize these settings in Fig. T0 We visualize the settings of Combination and Category in Fig. and Fig. 12
respectively.
In Distractor, we add distractor objects that are visually similar to at least one target object to the scene. In Lighting, we evaluate the model in novel lighting condition which the lights are off. In Background,
three different tablecloths to change the background. For Height, we elevate all objects for manipulation with a drawer that is about 10cm high. Distractor, Lighting, Background, and Height aim evaluate the robustness against visual disturbances.
In Combination, we combine objects and skills that are not paired together in the training datasets. That is, while the object for manipulation and the manipulation skill are seen during training, the instruction that pairs them together is novel. The setting of Combination helps us evaluate whether the model is able to generalize across novel object-skill combinations. In Category, want to evaluate whether Bridge is able to manipulate objects from categories that are unseen in the robot training data, In total, we test novel objects.
C.4 Preservation of Object Grounding Capability after Fine-tuning
We observe that even after fine-tuning on robot action data, BridgeVLA retains the object grounding capability learned during pre-training. We visualize its predictions on the pre-training dataset after fine-tuning in Fig. 14 It is important to note that the samples Fig. [4 are not cherry-picked. Bridge LLA does not forget its pre-training knowledge after 3D action fine-tuning.
25

---

C.5 Per-task Success Rate
We showcase per-task success rates of BridgeVLA the basic setting in Tab. 12 Notably, BridgeVLA
achieves exceptionally high success rates even with only 3 trajectories per task, highlighting its superb sample efficiency.
500
NO COLAR
Tate COl
Thak Nazk
310
beskstalirs Jxep
40:53
94
1035500
273019
(loe kox
1000000
723:80
947=19
06.311
1005:00
003100
96.=19
1000000
923511
1C::00
1002200
1142157
6352146
M05151
913331
137265
550033
261100
00::1
1,215
13219
48:53
00230
01200
00200
02230
13215
pru bokk
947130
53519
167.13
713-19
917419
65:31
507-112
maLie
958250
53813
320m0
10002000
1030500
900553
300219
raree Jenge
033:33
12231
520000
800:11
31R:57
33065
313:50
12:119
210100
960250
052233
933
WA7115
380133
933213
167219
1009500
547219
5582337
1135136
153577
120353
08:1:11
927:33
907230
913215
147255
520833
120236
947139
67658
117.13
147-13
003150
120113
113
S40-13
933119
967419
woay qusids
007139
Emand
13119
83100
133138
120157
313413
13355
53119
33457
00-13
sák. Hec begel
1000:00
203133
11::19
900
1015:00
003000
96.219
1000000
R0Be13
567238
213119
667211
307.13
460133
627415
68213
833551
267275
113:31
728136
013138
18257
198257
403
613394
63311
343252
313250
323232
728833
013119
153111
97--1
90.410
903131
94.275
Table 6: Success Rates of BridgeVIA under Different Perturbations of COLOSSEUM.
No aareaa
MO-T LAME
DO TEXTURE
MO- SIZE
103 102-0222
RLBeooh
RO
hoop
clou
96:49
11:1167
40:33
91252
$4:25
78:35
9129.1
98:28
V8135
96:28
95133
chooc_jptoppid
30:45
48:75
50:45
44200
33
enge, diswwoster
0:00
0:0.0
0:00
0000
1:17
1817
1817
0400
9:00
0400
0:00
0500
1:1.7
1:17
from, trigg
66:45
3120
71-333
44-24
31000
15-17
71:44
05:53
10155
09117
71144
bockey
12:28
1=60
2635
16:150
19617
moar grll
45:17
67-45
35417
64:34
61417
65-17
85153
07-11.1
63453
62-45
51433
Junger
0100
0:00
0:000
0100
0:00
0100
01000
0100
0:00
0100
01:00
01:00
0:00
0200
7777
9:59
1005000
side bicc taget
300500
37:107
100:00
100:000
101-00
91:1.
90:128
101:00
100:00
100-000
and
86:20
111.7
31.10
M128
25
14145
95117
79.11.
20:00
24100
12:152
451
11117
75117
M1128
jod_mwayyjtt_ad
63:17
121.
62:10
5:17
$9:10
75a1.7
64:28
60:00
47133
92.4.5
60:4.9
50100
80:28
place_ www. Jocaioo
96:49
Sha114
94:45
94820
9622
91217
81=25
$7459
99544
94s2.
30:160
18:28
95:33
96549
square Jeg
5:1.7
9:33
16:00
4100
4:00
exck cps
41100
3435
40-10
90:4.
8:00
10/00
19:17
10:60
15:33
40100
16:00
24100
$4:20
0:00
32:00
57:44
77:17
51:4A
142118
27:203
6121.7
56420
59217
choss
5833
0600
4:28
7433
4:25
12528
15452
soep with suada
96-00
040.
11-17
73433
534342
85-17
81417
81-53
57417
87-33
87444
574118
91417
85:33
90128
55.03
15:18
42.05
25102
99.16
51118
54110
$1.13
45103
41100
51123
46122
56.05
$4.05
54104
Table 7: Success Rates of RVT-2 under Different Perturbations of COLOSSEUM.
26

---

Methol
Aug.
Frigee0
Close
Jar-15
Closc
Jar+16
Close
Chosel
Lid+0
Microwareet
Class
LightBulh
Ins17
LightBulh
In-19
Bocro
Open
Doec+)
Ojec
Deswer-tl
Opea
Hivelomeet
60.3
64.11.5
92.27
58176
12:45
13:6.
A142
531:53
15:124
PourrNet
77800a
90113
99.23
19.22
135
98.17
72.125
71463
32am1
6916a
30 difirer
58.2x
RVT
SO-LO [P
100,00
108
3D Le
HH
68.7.n
100,00
99
28
$7
25
45.m0
$5
79
BLA (Oups)
91..11
9915
98.43
100.0.
90.05
55153
90,15
87175
76136.2
70:223
86.5.1
Method
Drswee11
Open
Fisk
Lihe0
Pick
Lifte2
Lite?
Piak
Pickip
Cip+5
Paztiip
Cupes)
Oxp+11
Psatip
Bomnee
Push
Humone3
Pish
Bononed
Push
Caphoando0
Putl
Hiveformet
59.74
M1:42
03a27
09.12
$1.11a
174
92a11
84.74
79
100
3D difleer
I0V
96
100
30-1 OOTS 1+
ghest
100.00
3D L
P2
75
97100
94.37
95151
8616a
5816
91.49
100.0.
100
100160
99.2
9i10
100.00
98.21
98.20
98137
99,30
100.000
98.41
98
74.6.3
Method
Pudlo Cighosnle3
PutMosey
16Safc4)
PubMones
Drsp-14
Reachd
Doge18
Reacha
Bleck40
Slide
Blxck+
Slide
Block-90
Stack
Blacks
Stack
Bladkse 19
Stack
Wa
PotaNNt
3D difluer
100
85
RVV2 [S]
85
100
17.6.7
98
3D-LONES [+
72
94.3.
99.23
99.20
100
100.10
100..0
94
91
3D Lor
DH
212.
22166
16:43
94.17
02:67
100.00
86.15
86158
20.41
28::36
UndgeVLA (Oin))
$4.31
79.97
10.57
97.5
100
90
77am1
87.41
Table 8: Per-task Success Rate on GemBench Level 1.
Method
Avg.
Buttone
Push
Button
Push
Button117
Push
Lift+14
Picka
Li
Picka
Life18
Picko
Cup-10
PickUp
Cupe12
PickUp
Cupe13
Piaktip
27.22
85.m0
21g
9232
8aur
22
PolarNer
27.1
100
100
87.7a
3445
Dino
48
46s8.
1646
3D difluser actor
43.4.2a
87 87.1
81
60.9.4
9442
18
Oino
84ast
60
62.
RVT-2
$1,0aaa
100.04
100
100 ..0
47a74
29.
8aas
8lasa
39.06
72,97
3D41 OTLOSUUS
49.9.9
99.30
100.54
100_44
3
18
Mana
89
78
57
LO-LOT
64.5
99
100
95.a
79
Bridge (Ours)
65.0
100
100
s.0
24
89.
O.no
91
90
.6.
Method
Blacks+24
Stack
Blockse
Stack
Blockse 33
Stack
Blocke2
Blocke3
Closs
Jara3
Close
Jare4
LightBulh
Insl
LightBulls
In=2
OnsO
Lamp
Hivelomer
4.42
Uaso
0asi
0ana
Osno
Dano
4u42
0aee
7
PolarNet
1aza
6as.1
0zaa
82a57
17a84
diffuser
662s9
82217
50a1c
0goe
23
82
60gus
18
361167
48
1aaa
7476
77att
68
61es
O1oa
3D-LOTUS
13s81
40.a1
69-1a
Osaa
71aaa
90.45
24.45
4lasc
3DJ KITUS4
22gs.
K3gra
39.17
27.ns
Sasa
98:2a
96.x
5622.7
43ata
2a20
BridgeVLA (Ours)
61 6171
79.44
12
3440
66166
88.40
60.s.
74
Reacha
Reacht
PuCube
Pick&Lin
Pick&Lift
Pick&Lin
Psck&Lin
Putln
Pulln
Mathod
Drag=5
Drag=7
InSafe40
Cylinder00
Moom+0
Toy+0
Cuphoard77
Cupboand48
Hivefomeer
1aza
0
4
78
73
88
87.4
Ogoa
Ogan
PolarNet
61asa
10aa.
40
93
88.44
93
90:35
0sea
Oann
3D diffuser actor
Osan
69ssa
321.
99.1.1
43
9lass
20.54
Osce
3:4b
RVT.2
91aza
892sa
98.17
9821a
94
78
Dznn
3D-LOTUS
18gms
25.5.
LOTUS+
94:20
64az24
37
91:12
94
294a8
7La20
Leza
Osnn
BridgeVi VLA (Qurs)
94
96217
Rezs
98223
99.2a
95
93
Ozau
Table 9: Per-task Success Rate GemBench Level 2.
27

---

Close
Close
Close
ClaselLapoop
Close
Av2.
Doo440
Box+0
Fridge240
Lid240
Microwave240
Door240
Box240
Hiveformer
35. 11.
0.00
1222
34200
$2.0.1
18
32-11.
S+35
PolarNet
2.7
Oauo
78a5.7
26awa
74.0.5
33
23
3D diffuser actor
37.0-22
0200
0.0.0
97=27
2320.7
8847.
86274
67458
RVT-2
36.0.22
1,2a
2
72187
42130.
71
79.85
S1s1
3D-LOTUS
1111
0200
50
36
S4
8547.1
42
3D-LOTUS++
41.
1,20
29
93125
50.05
99
52
161s0
BLd (Ours)
43.
020.
122.
98
77.4.0
54g30.2
68_
Open
Open
OpenDrawer
OpenDrawer
OpenDraweer Longs2
OpenDrawer
Toilet
Open
Method
Drawer20D
Drawer310
Lange0
Longel
Longe3
Seutlpp00
Friges0
Hiveformer
39-11
39
78:4.4
82:45
49.4.2
574115
6142
0zau
PolarNet
94242
29.8.2
842119
88g5.
63.8
37420
2
4
diftuser
19
1:za
15as.
35.
26
79,329
7
RVT-2
81211.
8428.2
39 39
1 Hes.
7846.1
745
0za0
3D-LOTUS
90.3u
22
56113
35
17 17.83
15.6.3
4158
3D-LOTUS
70
41.4.
7244.
52a10.
23
7845.1
8
0zan
Bridge VLA (Ours)
65
87
59 1s.s
34
18 181na
85
61s8
7125
Method
OpenLaptop
Lid+0
Microwave+0
Open
PutMoney
InSafc+2
Drawere1
Open
Drawer40
Close
Gril++0
Close
Hiveformer
100
0_0
010.
83
44.30.
PolarNet
100_u2
0.wa
1a2a
4242
29
42:11.5
3D difluser actor
100 100_n0
0_00
2145
0ia0
66g7.
661
RVT-2
93
0.wo
020.
6122
9.4.2
3D-LOTUS
100_00
0200
0.0.0
87.8.1
29
3D-LOTUS++
86164
13
69....
19.
BridgeVLA VLA (Ours)
95
0.00
242.5
020.0
58.12.
36g.2.3
Table 10: Per-task Success Rate GemBench Level 3.
TakeShnes OutOfBox40
Method
Avg.
Buttons4+1
Buttons4e2
Buttons443
InDrower40
InDruwer+2
Hiveformer
020.0
Ogn.o
020.0
Ogno
0g00
D200
PolarNet
0.120.
1a22
020.0
Oga.0
020.
3D diffuser actor
Ogoa
Osno
0.0.
Oguo
Os00
O200
RVT-2
040.0
Osn.o
0.0.0
O.a.
Osoe
040o
3D-LOTUS
0.3
3240
0go
Ogn
O0o.
O20o
0400
3D-LOTUS+
17.4
7627.4
49
37 37
O
Ozo.
0:0.
BridgeVLA (Ours)
Ogo.0
Ogn.o
0
Ogn.
Ogoo
Ogo.
0gno
Putltems
Stack
Stack
PutAllKimceries
Method
InDrawer44
Tower4+1
Tower4+3
Cups+0
Cups+3
InCupboard-0
020.0
Osn.o
020.0
040.0
0_0.0
0.0.0
PolarNet
Ogon
Oan.o
020.
Ogn
O2no
Oano
3D diffuser actor
0z0.0
Ozn.o
0.0.0
Ozno
O.c.
O.o.
RVT-2
0.0.
Osne
0go
Ogn.0
Ogno
Ogno
3D-LOTUS
040.0
Osn.o
0.:.0
Ozn.0
0=c.o
0s0.0
3D-LOTUS+
040.0
17
30.12.4
Oan
0=0.0
0=0.0
BridgeVLA (Ours)
040.0
Ogn.o
0.0.0
Ogn.0
0.0.0
0g0.0
Table 11: Per-task Success Rate GemBench Level 4.
28

---

Close jar
Insert onto square peg
Light blub in
Meat off grill
Place shape in shape sorter
Place wine at rack location
Open drawer
Push buttons
Place cups
Put groceries in cupboard
Put item in drawer
Put money in safe
Reach and drag
Slide block to color target
Stack cups
Sweep to dustpan of size
Turn tap
Stack blocks
Figure 5: Visualization of 18 RLBench [19 Tasks.
29

---

Basketball hoop
Close
Close laptop
All Variations
Empty dishwasher
Get ice from fridge
Insert onto square peg
Manipulation-
Object-Color
Hockey
Meat on grill
Place wine rack location
Receiving-
Object-Color
Put money in safe
Reach and drag
Scoop with spatula
Manipulation-
Object-Texture
Hockey
Insert onto square
Place wine rack location
Receiving-
Object-Texture
Setup chess
Wipe
Manipulation- Object-Size
Baskethall in hoop
Hockey
Insert onto square peg
Receiving-
Object-Size
Stack cups
Strengthen rope
Turn oven on
Light-Color
Put money safe
Reach and drag
Scoop with spatula
Table-Color
Close laptop lid
Get ice from fridge
Open drawer
Table-Texture
Move hanger
Setup chess
Slide block to target
Distractor
Basketball in hoop
Open drawer
Wipe desk
Backgroun d-Texture
Close box
Empty dishwasher
Setup chess
Camera Pose
Figure 6: Visualization of Perturbations
COLOSSEUM 35
30

---

Task
3 trajectories
10 frajectories
Put the RedBull can in the top shelf Put the soda can in the bottom shelf Put the RedBull can in the bottom shelf
9/10
10/10
9/10
9/10
10/10
10/10
Put the coke in the top shelf Place the red block in the blue plate
10/10
10/10
10/10
10/10
Place the orange block in the green plate
10/10
10/10
Put the wolf in the upper drawer
7/10
9/10
Place the red block the purple plate
10/10
10/10
Place the yellow block in the green plate
10/10
10/10
Press sanitizer
10/10
10/10
Put the zebra in the upper drawer Put the giraffe in the lower drawer Put the zebra in the lower drawer
9/10
9/10
10/10
9/10
10/10
10/10
Table 12: Per-task Success Rates of Bridge in the Basic Setting,
31

---

Place orange block in green plate
Initial Scene
Place red block in blue plate
Initial Scene
Place red block in purple plate
Initial Scene
Place yellow block in green plate
Initial Scene
Press sanitizer
Initial Scene
Put soda can in bottom shelf
Initial Scene
Put coke can in top shelf
Initial Scene
Figure 7: Real-Robot Rollouts (I).
32

---

Put Redbull can in bottom shelf
Initial Scene
Put Redbull can in top shelf
Initial Scene
Put giraffe in lower drawer
Initial Scene
Put wolf in upper drawer
Initial Scene
Put zebra in upper drawer
Initial Scene
Put zebra in lower drawer
Initial Scene
Figure 8: Real-Robot Rollouts (II).
33

---

haif eaten frested donut behind cup
horse in the back
glasses playing wil
orange grapes
person taking photo
right
small lamb sitting on ground on the right hand side next to two others.
the woven place malt
tractor with orange
white bowl with vegetables.
white daughnut right
white keyboard
zebra facing number three
elephant
Find all instances of circular frame with spokes
baby
far middle right elephant
Find all instances of decorative arrangement flowers.
Find all garments from waist to knee
ankle, covering each separately
Find instances bike
Find Instances of boot
Find instances clock
Find all instances
Find instances
greenest banana
holding one or more electric light buibs
vegotables
all Instances street sign
the woven place matt
Figure 9: Visualization of Pre-training Data. We list some samples of pre-training data. For every sample, the left shows the original image; the middle shows the bounding boxes of the objects of interest; the right shows the ground-trun heatmap used for training.
34

---

Task
Distractor
Lighting
Background
Height
Put the RedBull the top shelf
Put the soda can the bottom shelf
Place the red block the plate
Place the orange block the green plate
Press sanitizer
Put the zebra upper drawer
Put the giraffe the lower drawer
Figure 10: Visualization of the Distractor, Lighting, Background, and Height settings.
35

---

put the wolf in the lower drawer
put the giraffe in the upper drawer
place the orange block in the purple plate
place the red block in the green plate
place the orange block the blue plate
place the yellow block in the blue plate
place the yellow block in the purple plate
put the soda can in the top shelf
Figure 11: Visualization of the Combination Setting (1). During training, the manipulated objects and skills are seen, but their combinations are unseen.
36

---

Put the red block in the bottom shelf
Put the orange block the lower drawer
Put the soda can in the upper drawer
Put the Redbull can in the green plate
Place the zebra in the blue plate
Visualization of the Combination Setting (IT). During training, the manipulated objects and skills are seen, but their combinations are unseen.
Figure 12:
37

---

Press mouse
Put the apple top shelf
Put the peach In the bottom shelf
Put the sneaker in the upper drawer
Put the panda in lower drawer
Place the bread in green plate
Place the bottle in the blue plate
Visualization of the Category Setting, In total, we evaluate on 7 objects from novel categories that are unseen during training.
Figure 13:
38

---

Find instances of round faster
Find all instances baggage
Find instances outlery
Find all instancess
Find instances of neckwwer
leying down
container
Find Incouroe long sube plate
alarm
ERE
Find al instances beddlng
white doughnut
Find instances of veil
Figure 14: Visualization of BridgeVLA's Prediction Pre-training Dataset after Fine-tuning. To simulate the multi-view inputs during fine-tuning, we repeat the input image three times and feed them into the fine-tuned model to generate heatmaps. For each sample, the first shows the input image; the second row shows the heatmap prediction; the third row shows the ground truth.
39


Reviews:
[{"review_id": "jSWfThSXdd", "reviewer": "Reviewer_Prgj", "summary": "BridgeVLA is a 3D Vision-Language-Action (VLA) model that enhances robot manipulation by efficiently integrating 3D data with pre-trained vision-language models. It projects 3D inputs into 2D images for compatibility with VLMs and uses 2D heatmaps for action prediction, ensuring spatial consistency. A scalable pre-training phase further improves performance. Experiments show BridgeVLA outperforms state-of-the-art methods, achieving higher success rates with better sample efficiency.", "questions": "Please refer to the weaknesses section above for details：\n 1.Absence of Ablation Studies\n 2. Lack of Comparison with State-of-the-Art VLA Models\n 3. Insufficient Experimental Validation of Cross-Modal Bridging", "limitations": "Yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "1. We would recommend that the authors consider including ablation studies on the 2D heatmap prediction component.\n2.  Lack of Comparison with State-of-the-Art VLA Models: The current approach, which involves predicting 2D Heatmaps prior to action generation, is conceptually similar to methods like ECoT. Despite the authors' claim of using 3D point cloud data, the actual input to the VLA appears to be RGB data, warranting a direct comparison with ECoT. Furthermore, the authors should benchmark their method against contemporary VLA algorithms in a real experimental setting. Relevant comparisons include Pi\\_0, Co-Act RDT-1B,OpenVLA.OpenVLA-oft, 3D Diffusion Policy, and ACT.Additionally, we hope the authors can explore the model's performance on tasks such as folding clothes and folding pants.\n3.  Insufficient Experimental Validation of Cross-Modal Bridging:The paper asserts that the proposed method bridges the gap between 2D image-input Vision-Language Models (VLMs) and 3D inputs. However, most of the algorithms used for comparison do not utilize VLMs as their backbone, thereby failing to adequately demonstrate this claimed advantage.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer，\n\nWe sincerely appreciate your detailed and valuable feedback. In response to the concerns raised, we provide clarifications and additional analysis below. Hope they can address your concerns.\n\n### **Concern1: Lack of ablation studies on heatmap prediction**\n\nIn our original approach, we avoid direct action prediction by first generating 2D heatmaps through a convex upsampling module. We then compute the target positions by projecting the 3D points in the workspace onto the three heatmaps and selecting the point with the highest mean probability.\n\nFor the ablation, we replaced the convex upsampling module (309M parameters) with a Transformer decoder of similar size (303M parameters) to directly predict target positions, and supervise this prediction by MSE loss. The other modules are keep fixed. We conducted hyperparameter grid search for optimal performance, and evaluated the ablated model on RLBench. The results are summarized below:\n\n| Model               | close jar | reach & drag | insert peg | meat grill | open drawer | place cups | place wine | push button | put cupboard |\n|---------------------|-----------|--------------|------------|------------|--------------|-------------|-------------|--------------|----------------|\n| BridgeVLA w/o heat  | 49.3±2.3  | 65.3±2.3     | 0.0±0.0    | 81.3±4.6   | 74.7±10.1    | 1.3±2.3     | 32.0±14.4   | 54.7±6.1     | 5.3±2.3       |\n| BridgeVLA           | **100.0±0.0** | **100.0±0.0**    | **88.0±2.8**   | **100.0±0.0**  | **100.0±0.0**    | **58.4±10.0**   | **88.0±2.8**   | **98.4±2.2**     | **73.6±4.6**      |\n\n| Model               | put drawer | put money  | light bulb | slide block | place shape | stack blocks | stack cups | sweep dustpan | turn tap  | Average |\n|---------------------|------------|------------|-------------|--------------|--------------|---------------|-------------|----------------|------------|---------|\n| BridgeVLA w/o heat  | 0.0±0.0    | 58.7±22.7  | 2.7±2.3     | 64.0±0.0     | 4.0±4.0     | 0.0±0.0       | 0.0±0.0     | 32.0±4.0       | 40.0±10.6 | 31.4    |\n| BridgeVLA           | **99.2±1.8**   | **99.2±1.8**   | **87.2±6.6**    | **96.0±2.8**     | **60.8±7.7**    | **76.8±8.7**      | **81.6±3.6**   | **87.2±1.8**       | **92.8±3.3**  | **88.2**    |\n\nAs shown, replacing heatmap prediction with direct position prediction reduces the average success rate from 88.2% to 31.4%, confirming the effectiveness of our heatmap-based design. Additionally, we found the ablated model was much harder to train and more sensitive to hyperparameters, requiring a batch size of 192 and careful learning rate tuning for training stability, whereas our original model converges reliably even with a batch size of 64.\n\nWe believe there are three main reasons for this outcome. First, heatmaps provide a denser supervision signal compared to three-dimensional position vectors, which facilitates more effective learning. Second, the process of deriving target positions through projecting 3D points onto heatmaps implicitly incorporates additional spatial priors into the pipeline, thereby reducing the learning difficulty for the model. Third, the 2D heatmap outputs have the same spatial structure as the 2D image inputs. This spatial alignment helps the model better capture the correspondence between input and output, ultimately leading to improved performance.\n\n---\n\n### **Concern2: Comparison with State-of-the-Art VLA Models**\n\nThank you for your constructive comments. We fully agree that including more relevant baselines would strengthen our conclusions. This concern aligns closely with **Concern 1 raised by Reviewer gMBw**, to which we have provided a detailed response earlier. Due to word limit, we avoid repeating these comparing experiments here. Please kindly refer to our response under Concern 1 raised by Reviewer gMBw.\n\n---\n\n### **Concern3: Comparison with ECoT**\n\nThanks for your valuable suggestion. We have carefully read the ECoT paper and completely agree that ECoT shares similarities with BridgeVLA, as both involve intermediate representations. We will cite and discuss with ECoT paper in the final version of our manuscript. However, we would like to highlight several key differences between the two approaches:\n\ni）ECoT and BridgeVLA are complementary rather than mutually exclusive. ECoT contributes primarily by enabling reasoning before action execution, while the final action output is generated using an existing manipulation policy such as OpenVLA — which could potentially be replaced by BridgeVLA. Conversely, ECoT’s core strategies—such as task decomposition and leveraging foundation models to generate bounding boxes and end-effector positions—could also be integrated into the BridgeVLA framework.\n\nii) In ECoT, intermediate results—such as sub-instructions and bounding boxes—are fed into a learnable downstream policy to enhance the accuracy of final action prediction. Therefore, these intermediate results are only used to aid other modules. In contrast, for BridgeVLA, once the 2D heatmaps are generated, it projects 3D points from the workspace onto the heatmaps and selects the point with the highest mean probability as the target position. This process is deterministic and non-learnable. Therefore, heatmaps in BridgeVLA are directly used to get target positions rather than aid other modules.\n\nSince we have already compared against π0, which is also a SOTA 2D VLA model, we do not include a direct comparison with ECoT in this version. We consider the integration of ECoT’s techniques with BridgeVLA a promising direction for future work.\n\n---\n\n### **Concern4: Performance on tasks such as folding pants**\n\nThank you for your insightful suggestion. We agree that incorporating experiments on folding clothes or pants would further validate the effectiveness of BridgeVLA. Since we use a single Franka Research 3 arm as our robotic embodiment, we are unable to perform the highly dexterous cloth-folding tasks demonstrated in π0 due to physical limitations. However, we have followed your suggestions and implemented a simplified version on our system—folding a pair of pants first from left to right, and then from front to back.\n\nTo evaluate our model’s performance, we define two metrics: **Task Success Rate (TSR)** and **Sub-Task Success Rate (STSR)**. For TSR, a trial is considered successful only if the robot completes both folding actions—left-to-right and front-to-back—within a single execution. For STSR, a partial score is given: 0.5 is awarded if the robot successfully folds the pants from left to right, and another 0.5 if it subsequently folds them from front to back.\n\nWe define success criteria as follows:\n- Left-to-right fold: Considered successful if the left side of the pants is placed beyond the midpoint.\n- Front-to-back fold: Considered successful if the front side is placed beyond the midpoint.\n\nThe experimental results are shown below:\n\n| Metric | Score  |\n|--------|--------|\n| TSR    | 5/10   |\n| STSR   | 7.5/10 |\n\nWe observe that the robot consistently succeeds in folding the pants from left to right. However, it only succeeds about half the time when folding from front to back. This discrepancy arises because the initial shape of the pants matches those seen in the training data, allowing the model to accurately predict the grasping location for the first fold. However, after the initial fold, the deformable nature of the pants combined with the limitations of single-arm manipulation leads to unpredictable object shapes. As a result, the robot often fails to locate or properly grasp the front side of the pants in the second step.\n\n---\n\n### **Concern5: Experimental Validation of Cross-Modal Bridging**\n\nWe greatly appreciate your valuable comments. As detailed in our response to Concern 2, we have conducted real robot experiments comparing BridgeVLA with SpatialVLA and π0—both of which employ large-scale VLMs as their backbones. Notably, both BridgeVLA and π0 use the same PaliGemma backbone, while SpatialVLA even adopts the more advanced PaliGemma 2 backbone.\n\nDespite this, the experimental results show that BridgeVLA significantly outperforms both π0 and SpatialVLA (97.5% vs. 5% and 2.5% , respectively). This highlights the effectiveness of our bridging design in more fully leveraging the capabilities of 2D vision-language models when the finetuning data is extremely limited."}, {"review_id": "d8fGunKrXD", "reviewer": "Reviewer_gMBw", "summary": "The paper proposed, BridgeVLA a 3D vision-language-action (VLA) model that projects 3D inputs into multiple 2D images and uses 2D heatmaps for action prediction, aligning both input and output spaces for efficient robot manipulation learning. The authors train the model in 2 stages:\n\na) pre-training the VLM backbone to predict heatmaps with task relevant points, and\nb) training a policy on top of the heat maps to predict the end-effector pose of the robot.\n\nBridgeVLA achieves state-of-the-art results on the RL-Bench benchmark and shows significant robustness on the COLLESUM benchmark.", "questions": "3. What is the inference time for the overall policy?\n\n4. In Table 4, what could be the reason that BridgeVLA performs on \"RO-TEXTURE\" than on \"NO Variations\"?\n\n5. There are no long-horizon tasks included in the evaluation (both in sim and real world). I'm wondering if the pipeline would still work when chained with multiple tasks. For instance, \"Put the Red Bull can on the bottom shelf and the Coke can on the top shelf.\" I hypothesize that the heatmap generation may fail (or maybe not) as it wasn't trained on such prompts. But any insights on this would be helpful.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n- The method is sound and novel and furthers the boundary of 2D heatmap-based robotic manipulation.\n- Leveraging the RoboPoint dataset for pre-training is a very neat way of learning 2D heatmap priors for the downstream task of manipulation.\n- Well-written paper.\n\n\n**Weaknesses**:\n\n1. Comparison to alternative 3D VLA method like FP3 is missing. I'm curious if the heatmap-based approaches perform significantly better than natively trained 3D policies. (PointVLA doesn't open-source their code, so a comparison there isn't required).\n\n2. ACT [1] has become a strong BC baseline in recent years, so I believe that having the ACT baseline the way the Diffusion Policy baseline (3D diffuser actor) was added would make this paper stronger.\n\n----\n\n**References**\n\n[1] Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware, Tony Zhao et al., RSS 2023", "was_revised": true, "final_justification": "Thanks to the authors for addressing all my concerns (adding VLA baselines, addressing differences between RO-TEXTURE and No-Variations, adding ACT baseline).\n\nI will continue to maintain my score of **Accept**.", "rebuttal": "Dear Reviewer,\n\nThank you very much for your valuable feedback. We address your concerns with the following clarifications and additional analysis, and hope our responses are helpful.\n\n### **Concern1: Comparison to other 3D VLA methods**\n\nWe sincerely thank you for the constructive feedback. In response, we conducted additional comparisons between BridgeVLA and three types of manipulation policies, including SpatialVLA as a 3D VLA method. These results will be included in the final version of the paper.\n\n1）SpatialVLA: A state-of-the-art **3D VLA** model that incorporates 3D information through Ego3D positional encoding and leverages Adaptive Action Grids to accelerate inference.\n\n2）π0: A state-of-the-art **2D VLA** model pretrained on a large-scale cross-embodiment dataset. It adopts a vision-language model (VLM) backbone and employs a diffusion-based action expert to generate final actions.\n\n3）ACT:  A state-of-the-art 2D **non-VLA** model using a Conditional Variational Autoencoder (CVAE) to model action distributions. Though effective for fine-grained manipulation, ACT does not support language conditioning, so we train a separate single-task model for each task, which should theoretically perform better than a multi-task version.\n\nAs for **3D non-VLA** models, we have already compared with leading methods such as RVT-2 and 3D Diffuser Actor in the paper, and thus do not repeat them here. \n\nDue to time constraints, we were unable to test all 13 real-robot tasks from the paper. Since some tasks are similar (e.g., \"Put the RedBull can in the bottom shelf\" vs. \"Put the soda can in the bottom shelf\"), we selected four representative tasks for comparison. As in the original setting, each baseline was trained with 10 trajectories per task and evaluated over 10 trials to ensure statistical robustness. For fair comparison, we photographed each test scene and manually aligned the setups across all methods.\n\nThe results are presented below, along with our observations:\n\n| Model           | Put the soda can in the bottom shelf | Put the giraffe in the lower drawer | Place the red block in the blue plate | Press Sanitizer | Average |\n|----------------|--------------------------------------|--------------------------------------|----------------------------------------|------------------|---------|\n| SpatialVLA-50  | 2/10                                 | 2/10                                 | 4/10                                   | 6/10             | 35.0%   |\n| SpatialVLA-10  | 0/10                                 | 0/10                                 | 0/10                                   | 1/10             | 2.5%    |\n| π0             | 0/10                                 | 0/10                                 | 1/10                                   | 1/10             | 5.0%    |\n| ACT            | 1/10                                 | 2/10                                 | 3/10                                   | 3/10             | 22.5%   |\n| BridgeVLA      | **10/10**                                | **9/10**                                 | **10/10**                                  | **10/10**            | **97.5%**   |\n\n\n\n**SpatialVLA**: According to the experimental setup, we initially trained SpatialVLA with only 10 trajectories per task. However, it failed on nearly all tasks, often struggling even to move toward the correct target object. Therefore, we augmented the dataset with an additional 40 trajectories per task. The performance improved but remained substantially lower than that of BridgeVLA—particularly on more challenging tasks such as *Put the giraffe in the lower drawer*. These findings suggest that BridgeVLA offers a more effective and data-efficient 3D VLA solution.\n\n**π0**: Similarly, π0 fails with only 10 trajectories per task, likely due to overfitting—it performs well on training inputs but often fails during online testing. Common failure modes include missing or failing to grasp the target, and prematurely opening the gripper before reaching the goal. Notably, both BridgeVLA and π0 share the same PaliGemma backbone and are trained end-to-end. This underscores a key contribution of our work: while VLAs like π0 perform well with large-scale data, **they struggle in low-data regimes**—even on simple tasks like Press sanitizer. In contrast, BridgeVLA achieves near-perfect success and generalizes robustly across diverse settings.\n\n**ACT**: ACT also underperforms compared to BridgeVLA. It shows limited spatial generalization—performing well only in areas densely covered during training, but often failing when the target is near workspace boundaries. This aligns with its design: ACT models actions with a Gaussian prior, which assigns low probability to peripheral regions, limiting spatial generalization.\n\nGiven that none of these baseline models can reliably solve the basic settings under such low-data conditions, we did not proceed with further evaluations on generalization or transfer scenarios.\n\n---\n\n### **Concern2: Comparison with ACT**\n\nThank you for your insightful suggestion. We have addressed this concern in our response to Concern 1.\n\n---\n\n### **Concern3: Inference time for the overall policy**\n\nWe greatly appreciate your constructive comments. We evaluated the inference speed of BridgeVLA by running it 100 times on a machine equipped with an NVIDIA RTX 4090 GPU. From point cloud input to action output, the average end-to-end inference time is 0.21 seconds.\n\n---\n\n### **Concern4: Why BridgeVLA performs better on \"RO-TEXTURE\" than on \"NO Variations\"?**\n\nWe sincerely apologize for the confusion caused by our earlier mistake. The original evaluation script contained some minor bugs, which have now been fixed. After correction, BridgeVLA performs  worse on the \"RO-TEXTURE\" setting (68.4%) compared to the \"NO Variations\" setting (73.9%).\n\nImportantly, this does not impact the main conclusions, as BridgeVLA still clearly outperforms all baselines on the COLOSSEUM benchmark. The correct success rate is provided below, and we will update the table in the final version.\n\n| Models      | Avg. SR (%) | Avg. Rank | All Perturbations | MO-COLOR | RO-COLOR | MO-TEXTURE | RO-TEXTURE | MO-SIZE |\n|-------------|--------------|------------|---------------------|------------|-------------|---------------|---------------|-----------|\n| R3M-MLP     | 0.8          | 5.71       | 0.6                 | 0.4        | 0.0         | 0.0           | 0.0           | 1.8       |\n| MVP-MLP     | 1.6          | 5.00       | 0.8                 | 1.2        | 0.0         | 0.4           | 0.0           | 4.44      |\n| PerAct      | 27.9         | 3.71       | 7.2                 | 24.0       | 29.2        | 28.8          | 17.71         | 35.6      |\n| RVT         | 35.4         | 3.28       | 6.4                 | 26.0       | 31.3        | 44.8          | 41.1          | 35.3      |\n| RVT-2       | 56.7         | 1.92       | 15.6                | 53.0       | 54.6        | 59.7          | 56.7          | 60.9      |\n| BridgeVLA   | **64.0**     | **1.07**   | **18.7 ± 2.2**      | **60.5 ± 1.1** | **63.8 ± 0.1**  | **63.5 ± 1.5**    | **68.4 ± 3.3**  | **69.3 ± 1.0** |\n\n| Models      | RO-SIZE | Light Color | Table Color | Table Texture | Distractor | Back-Texture | RLBench | Camera Pose |\n|-------------|---------|-------------|--------------|----------------|-------------|----------------|----------|---------------|\n| R3M-MLP     | 0.0     | 1.0         | 1.4          | 0.2            | 1.6         | 1.2            | 2.0      | 0.8           |\n| MVP-MLP     | 0.0     | 1.6         | 1.6          | 1.0            | 3.8         | 2.2            | 2.0      | 2.6           |\n| PerAct      | 29.3    | 29.1        | 30.4         | 23.2           | 27.1        | 33.5           | 39.4     | 36.3          |\n| RVT         | 40.5    | 34.0        | 30.0         | 45.2           | 18.8        | 46.4           | 53.4     | 42.2          |\n| RVT-2       | 53.4    | 58.0        | 62.6         | 56.6           | 60.8        | 68.7           | 68.8     | 64.4          |\n| BridgeVLA   | **61.7 ± 0.8** | **69.7 ± 1.2** | **75.7 ± 0.9**  | **71.3 ± 0.7**     | **51.8 ± 1.5**  | **74.8 ± 1.0**   | **73.1 ± 0.2** | **73.8 ± 0.3**  |\n\n---\n\n### **Concern5: Insights on long horizon tasks**\n\nWe are deeply grateful for your thoughtful suggestion regarding long-horizon tasks. We evaluated BridgeVLA on the examples you provided and found it currently cannot handle them effectively. However, several interesting patterns emerged.\n\nWe ran two instructions 10 times each:  \n- “Put the RedBull can on the bottom shelf and then put the Coke can on the top shelf.”  \n- “Put the Coke can on the bottom shelf and then put the RedBull can on the top shelf.”   \n\nIn all 20 trials, BridgeVLA was consistently able to place one can on the shelf successfully. However, after placing the first can, the robot failed to proceed to the second part of the task, instead moving back and forth near the endpoint without further action. This behavior is understandable, as the model was never trained to return and pick up a second object.\n\nInterestingly, we observed that in 12 out of 20 trials, the model placed the correct can on the correct shelf, while in 8 trials, it did not. Regarding the execution order, the model followed the sequence described in the instruction in 11 out of 20 trials.\n\nThis suggests BridgeVLA can recognize object entities but struggles with relational reasoning and temporal sequencing. We believe this is due to our 2D heatmap pretraining, which was based on a 120K object detection dataset. While it helps with object recognition, it lacks exposure to higher-level semantics such as object-location mapping or multi-step task structure.\n\nTo address this, we plan to explore pretraining strategies using more semantically diverse datasets to improve BridgeVLA’s ability to interpret and execute complex, multi-step instructions."}, {"review_id": "7gEOCyJjee", "reviewer": "Reviewer_Qejn", "summary": "BridgeVLA is a 3D vision-language-action (VLA) model that addresses the misalignment between 3D inputs and 2D pre-trained vision-language models(VLMs). BridgeVLA first pre-trains a vision-language models to predict 2D object heatmaps on standard images, then fine-tunes it on 3D robot data by rendering RGB-D point clouds into three orthographic views and using the resulting heatmaps to drive end-effector translation, rotation, gripper, and collision predictions. This approach not only reconciles the 3D-2D modality gap but also outperforms prior 3D manipulation methods - achieving higher success rates on RLBench benchmarks.", "questions": "The questions below extend the weakness outlined above:\n\n1. Prior work (e.g., HAMSTER[1]) already pre-trains on 2D images and then uses a 3D policy. How does BridgeVLA differ from HAMSTER?\n2. HAMSTER's([1]) 2D pre-training also includes a task for the VLM to predict object pixel locations in the image. Compared to that, what makes BridgeVLA's heatmap pre-training more spatially aware?\n3. In Table 1, can BridgeVLA be directly compared to other VLA methods on RLBench? Could you show a head-to-head comparison where OpenVLA[2], π₀[3], HAMSTER[1], etc., are fine-tuned using the same amount of robot data?\n4. Inference seems potentially slow - what is BridgeVLA's actual inference time (in HZ)?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. Unified 2D Alignment : By learning a shared 2D heatmap representation in both pre-training and fine-tuning, BridgeVLA aligns 2D features with 3D robot data.\n2. Data Efficiency : BridgeVLA achieves strong task performance with very limited robot demonstrations (3 trajectories per task)\n3. Robustness & Generalization : BridgeVLA demonstrates resilience under visual disturbances and generalizes to novel objects and unseen object-skill combinations.\n\nWeakness:\n\n1. No Direct Baseline Comparison : The paper does not report head-to-head performance against other VLA methods (e.g., OpenVLA[2], π0[3], HAMSTER[1]).\n2. Limited Novelty : Prior works(HAMSTER[1]) already pre-train vision-language models on 2D images to learn spatial representations that a downstream 3D policy can directly leverage, and some even train the VLM to predict end-effector trajectories in the image plane. BridgeVLA's main technical departures are its specific 2D heatmap - based pre-training architecture, its orthographic projection and the addition of a motion planner after action prediction. To make the contribution clearer, the authors should demonstrate how their approach differs from prior works([1],[2],[3]) and pinpoint which components-such as heatmap prediction, orthographic projections, or the post-prediction motion planner-specifically drive the observed performance improvements.\n3. Unreported Inference Speed : Inference latency isn't measured. Real-time applicability remains underexplored.\n\n[1] Li, Yi, et al. \"Hamster: Hierarchical action models for open-world robot manipulation.\"arXiv preprint arXiv:2502.05485 (2025).\n\n[2] Kim, Moo Jin, et al. \"Openvla: An open-source vision-language-action model.\"arXiv preprint arXiv:2406.09246 (2024).\n\n[3] Black, Kevin, et al. \"π0: A vision-language-action flow model for general robot control, 2024.\"URL[https://arxiv](https://arxiv/). org/abs/2410.24164", "was_revised": true, "final_justification": "Most of my concerns and questions have been addressed.", "rebuttal": "Dear Reviewer,\n\nThank you very much for your time and thoughtful feedback. Below, we provide further clarifications and additional analysis in response to your concerns, and we hope our responses address them satisfactorily.\n\n### **Concern1: Difference between HAMSTER and BridgeVLA**\n\nThank you for your constructive comments. We completely agree that HAMSTER shares similarities with BridgeVLA as they both pretrain on 2D images. However, they also differ in several key aspects:\n\n**Architecture Design:**\n- HAMSTER adopts a **hierarchical architecture**, where a frozen VLM generates 2D image-space paths, which are then interpreted by a separate low-level 3D policy to output actions.\n- BridgeVLA is a fully **end-to-end** model that uses a VLM as the backbone. All components, including the VLM, are trained jointly using action supervision.\n\n**Training Strategy:**\n- In HAMSTER, the vision-language model (VLM) is frozen during finetuning, and the action loss only updates the low-level manipulation policy. Additionally, its VLM pretraining predicts object pixel locations by autoregressively generating language token sequences. These sequences are temporally ordered but lack intrinsic spatial structure.\n- In BridgeVLA, it adopts an end-to-end training strategy where the action loss updates all model parameters, including those of the VLM. Its heatmap pretraining directly produces 2D heatmaps with explicit spatial structure and each heatmap pixel corresponds one-to-one with a pixel in the input image. \n\n**Compounding error:**\n- HAMSTER is prone to compounding error, as mistakes in either the VLM’s path prediction or the low-level policy can cause task failure.\n- BridgeVLA avoids such compounding error as heatmap-to-position process uses a deterministic projection from 3D points to image planes and selects the point with the highest probability on the heatmap. This step is precise and introduces no learning-related uncertainty.\n\n**Integration Possibility:**\n- BridgeVLA can serve as an improved low-level policy within HAMSTER’s hierarchy. It transforms point clouds into multi-view images, on which we can draw image paths using HAMSTER’s pretrained VLM.\n- In HAMSTER’s original implementation, RVT-2 and 3D Diffuser Actor are used as the low-level policy. As shown in our experiments, BridgeVLA significantly outperforms both, and we believe its integration could further enhance HAMSTER's performance.\n\n---\n\n### **Concern2: What makes BridgeVLA's heatmap pretraining more spatially aware than HAMSTER's VLM pre-training?**\n\nThank you for your question. We think it is very insightful and important. \n\nHAMSTER's VLM pretraining predicts object pixel locations through autoregressive generation of language token sequences. Although temporally ordered, these sequences lack intrinsic spatial structure. The predicted tokens must be decoded into numeric symbols before they can be associated with pixel coordinates, introducing additional abstraction that weakens spatial grounding.\n\nIn contrast, BridgeVLA's heatmap pretraining predicts 2D heatmaps that maintain explicit spatial structure. Each heatmap pixel corresponds one-to-one with a pixel in the input image, enabling strong spatial alignment. This direct spatial correspondence allows BridgeVLA to learn more spatially grounded representations, making it more spatially aware than HAMSTER’s VLM pretraining.\n\n---\n\n### **Concern3: Comparison with other VLA methods**\n\nWe greatly appreciate your constructive comments. It aligns closely with **Concern 1 raised by Reviewer gMBw**, to which we have provided a detailed response earlier. Due to word limit, we avoid repeating this comparing experiments here. Please kindly refer to our response under Concern 1 raised by Reviewer gMBw.\n\n---\n\n### **Concern4: Inference time of BridgeVLA**\nThank you for your constructive comments. We evaluated the inference speed of BridgeVLA by running it 100 times on a machine equipped with an NVIDIA RTX 4090 GPU. The average end-to-end inference time—from point cloud input to action output—is 0.21 seconds, corresponding to approximately 5 Hz.\n\n---\n\n### **Concern5: Ablations on BirdgeVLA's main components**\n\nWe are deeply grateful for your constructive comments. In our paper, we conducted ablation studies on pretraining technique. Our real-robot experiments demonstrate that our pretraining significantly improves BridgeVLA's generalization ability, especially under the Combination setting, where the average success rate increases from 5% to 64%. Below, we provide additional clarification regarding the three components you mentioned and we will also add these experiments into our final version.\n\n**1) Heatmap Prediction**\nIn our original approach, we avoid direct action prediction by first generating 2D heatmaps through a convex upsampling module. We then compute the target positions by projecting the 3D points in the workspace onto the three heatmaps and selecting the point with the highest mean probability.\n\nFor the ablation, we replaced the convex upsampling module (309M parameters) with a Transformer decoder of similar size (303M parameters) to directly predict target positions, and supervise this prediction by MSE loss. The other modules are keep fixed. We conducted hyperparameter grid search for optimal performance, and evaluated the ablated model on RLBench. The results are summarized in Table below.\n\nAs shown, replacing heatmap prediction with direct position prediction reduces the average success rate from 88.2% to 31.4%, confirming the effectiveness of our heatmap-based design. Additionally, we found the ablated model was much harder to train and more sensitive to hyperparameters, requiring a batch size of 192 and careful learning rate tuning for training stability, whereas our original model converges reliably even with a batch size of 64.\n\nWe believe there are three main reasons for this outcome. First, heatmaps provide a denser supervision signal compared to three-dimensional position vectors, which facilitates more effective learning. Second, the process of deriving target positions through projecting 3D points onto heatmaps implicitly incorporates additional spatial priors into the pipeline, thereby reducing the learning difficulty for the model. Third, the 2D heatmap outputs are defined in the same spatial domain as the 2D image inputs. This spatial alignment helps the model better capture the correspondence between input and output, ultimately leading to improved performance.\n\n**2) Orthographic Projection**\nIt is not feasible to ablate orthographic projection in isolation. Without multi-view projections, we cannot compute 3D target positions from 2D heatmaps, and the entire pipeline would fail.\n\nMeanwhile, orthographic projection itself is not the main focus—it is primarily a tool we use to convert 3D point clouds into 2D images. This transformation ensures that the input feature space during fine-tuning is well aligned with the VLM pretraining space, which we believe is crucial for building a unified vision-language-action (VLA) model.\n\nTo test the importance of this alignment, we conducted an additional ablation inspired by SpatialVLA. Specifically, we added a 3D convolutional module to encode 3D positions of each image patch, which were then fused with the corresponding 2D image features before feeding into the backbone. While this introduces richer spatial information, it significantly shifts the distribution of image features compared to those seen during VLM pretraining. As a result, although we did hyperparameter grid search to get optimal performance, it still dropped from 88.2% to 56.2% on RLBench. Detailed results are shown in Table below.\n\n**3) Motion Planner**\nBridgeVLA is a keypoint-based method, and all such methods have to rely on a motion planner to move between predicted keypoints. As such, this component can not be regarded as our contribution and cannot be meaningfully ablated in isolation.\nIn our experiments, we simply use default motion planners—RRT-Connect. If you had a specific type of ablation in mind regarding this module, we would greatly appreciate it if you could clarify your expectations.\n\n| Model               | close jar | reach & drag | insert peg | meat grill | open drawer | place cups | place wine | push button | put cupboard |\n|---------------------|-----------|--------------|------------|------------|--------------|-------------|-------------|--------------|----------------|\n| BridgeVLA w/o heat  | 49.3±2.3  | 65.3±2.3     | 0.0±0.0    | 81.3±4.6   | 74.7±10.1    | 1.3±2.3     | 32.0±14.4   | 54.7±6.1     | 5.3±2.3       |\n| BridgeVLA w pos     | 96.0±0.0  | 58.7±6.1     | 26.7±2.3   | 96.0±0.0   | 97.3±2.3     | 14.7±4.6    | 81.3±8.3    | 86.7±2.3     | 10.7±2.3      |\n| BridgeVLA           | **100.0±0.0** | **100.0±0.0**    | **88.0±2.8**   | **100.0±0.0**  | **100.0±0.0**    | **58.4±10.0**   | **88.0±2.8**   | **98.4±2.2**     | **73.6±4.6**      |\n\n| Model               | put drawer | put money  | light bulb | slide block | place shape | stack blocks | stack cups | sweep dustpan | turn tap  | Average |\n|---------------------|------------|------------|-------------|--------------|--------------|---------------|-------------|----------------|------------|---------|\n| BridgeVLA w/o heat  | 0.0±0.0    | 58.7±22.7  | 2.7±2.3     | 64.0±0.0     | 4.0±4.0     | 0.0±0.0       | 0.0±0.0     | 32.0±4.0       | 40.0±10.6 | 31.4    |\n| BridgeVLA w pos     | 78.7±2.3   | 97.3±4.6   | 16.0±4.0    | 72.0±0.0     | 21.3±8.3    | 17.3±2.3      | 4.0±4.0     | 53.3±2.3       | 84.0±0.0  | 56.2    |\n| BridgeVLA           | **99.2±1.8**   | **99.2±1.8**   | **87.2±6.6**    | **96.0±2.8**     | **60.8±7.7**    | **76.8±8.7**      | **81.6±3.6**   | **87.2±1.8**       | **92.8±3.3**  | **88.2**    |"}, {"review_id": "0d69Sy3ogb", "reviewer": "Reviewer_J4oj", "summary": "This paper presents BridgeVLA, a 3D vision-language-action (VLA) model that leverages the structural priors of 3D data (i.e., point clouds) while mitigating a large distributional shift from the original vision-language model training. A key idea of BridgeVLA is to transform 3D point cloud observation into multiple 2D images and further learn to predict a 2D heatmap highlighting regions of interest. 2D heatmaps are used to predict robotic actions. Experiments show that BridgeVLA outperforms several baselines on simulated environments (RLBench and COLOSSEUM) and real-world environments.", "questions": "N/A", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- (S1) The manuscript proposes a new 3D VLA model for robotic tasks \n- (S2) The paper conducted extensive analyses across real-world and simulation evaluations\n- (S3) The paper is generally well-written\n\n**Weaknesses**\n\n- (W1) **Limited empirical justification for core claims**. The current manuscript mainly argues:\n  - (1) Current 3D VLA models [1,2] are sample inefficient since they do not fully exploit the spatial structure inherent in 3D data.\n  - (2) Using 3D data for fine-tuning models trained on 2D inputs introduces a large distributional shift that harms performance.\n- However, the authors did not validate these critical claims in their experiments. They only compared BridgeVLA to non-VLA baselines (e.g., RVT-2), without benchmarking against any existing 3D VLA models. This omission weakens the empirical support for their critique of prior work and leaves the practical advantages of the proposed method unsubstantiated.\n\n- (W2) **Lack of component-level analysis**. The paper claims superior performance over prior work across diverse environments, but it does not provide a deeper investigation into the sources of these improvements. Without detailed ablations or analysis isolating the contributions of individual design choices, it is difficult for the community to interpret where the gains come from or derive actionable insights for future work.\n\n- (W3) **Lack of clarity in core components**. Some technical details of the proposed method are insufficiently described. In Chapter 3, key components of BridgeVLA—such as convex upsampling and the loss functions—are not presented with enough precision. This lack of clarity hinders reproducibility and makes it difficult for readers to fully understand the technical underpinnings of the approach.\n\n**References**\n- [1] 3D-VLA: A 3D Vision-Language-Action Generative World Model. ICML 2024.\n- [2] SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model. RSS 2025.", "was_revised": true, "final_justification": "In the rebuttal phase, the authors addressed my concerns regarding clarity, component-level analysis, and empirical justification. While they have made notable progress, the manuscript still requires significant effort to improve the clarity and depth of analysis of the newly added experiments (e.g., explaining the large performance gap compared to SpatialVLA and π0) before it can meet the bar for publication. Therefore, I am finalizing my rating as borderline reject.", "rebuttal": "Dear Reviewer,\n\nWe greatly appreciate your careful review and valuable suggestions. In the following, we offer detailed clarifications and additional analysis in response to your comments, and we hope these address your concerns.\n\n### **Concern1:  Limited empirical justification for core claims**\n\nWe are deeply grateful for your constructive comments. To demonstrate BridgeVLA’s advantages, we compared it with SpatialVLA, a state-of-the-art 3D VLA method. As the word count is limited and a detailed comparison has already been provided in our response to **Concern 1 by Reviewer gMBw**, we avoid repeating it here and kindly refer you to that reply.\n\n---\n\n### **Concern 2: Lack of component-level analysis**\n\nThank you for your insightful comments. Our paper includes ablations on the pretraining technique, showing it greatly improves BridgeVLA’s generalization—especially in the Combination setting (from 5% to 64% success). Your suggestion goes a step further, and we agree that more ablations are valuable for understanding the source of these gains. Below, we present two additional studies, which will be included in the final version.\n\n**1) Whether we need to predict heatmaps before predicting actions**\n\nOur approach avoids direct action prediction by first generating 2D heatmaps using a convex upsampling module. Target positions are then computed by projecting 3D workspace points onto the heatmaps and selecting the point with the highest mean probability.\n\nFor ablation, we replaced the convex upsampling module (309M parameters) with a similarly sized Transformer decoder (303M) to directly predict target positions, supervised by MSE loss. All other modules were kept fixed. We performed a hyperparameter grid search and evaluated the model on RLBench. Results are shown in the table below.\n\nReplacing heatmap prediction with direct position regression reduced the average success rate from 88.2% to 31.4%, confirming the effectiveness of our heatmap-based design. The ablated model was also harder to train and more sensitive to hyperparameters—requiring a batch size of 192 and careful learning rate tuning—while our original model trains reliably with a batch size of 64.\n\nWe see three main reasons for this outcome: (1) Heatmaps offer denser supervision than 3D position vectors, enabling more effective learning. (2) Projecting 3D points onto heatmaps introduces helpful spatial priors, easing the learning process. (3) The 2D heatmaps share the same spatial structure as the input images, enhancing alignment and improving performance.\n\n**2) Whether we need to remove the 3D position input to the VLM backbone**\n\nUnlike typical 3D VLA models like SpatialVLA, we deliberately **avoid** using per-pixel 3D position inputs and rely solely on RGB images. This design preserves alignment between the input feature spaces of fine-tuning and VLM pretraining, which we find crucial for effective vision-language-action (VLA) modeling.\n\nTo test this, we added a 3D convolutional module to encode per-pixel 3D positions, fused them with 2D features, and fed the result into the backbone. Although this adds richer spatial cues, it alters the image feature distribution seen during pretraining, leading to a performance drop from 88.2% to 56.2% on RLBench. Detailed results are shown in the table below.\n\n| Model               | close jar | reach & drag | insert peg | meat grill | open drawer | place cups | place wine | push button | put cupboard |\n|---------------------|-----------|--------------|------------|------------|--------------|-------------|-------------|--------------|----------------|\n| BridgeVLA w/o heat  | 49.3±2.3  | 65.3±2.3     | 0.0±0.0    | 81.3±4.6   | 74.7±10.1    | 1.3±2.3     | 32.0±14.4   | 54.7±6.1     | 5.3±2.3       |\n| BridgeVLA w pos     | 96.0±0.0  | 58.7±6.1     | 26.7±2.3   | 96.0±0.0   | 97.3±2.3     | 14.7±4.6    | 81.3±8.3    | 86.7±2.3     | 10.7±2.3      |\n| BridgeVLA           | **100.0±0.0** | **100.0±0.0**    | **88.0±2.8**   | **100.0±0.0**  | **100.0±0.0**    | **58.4±10.0**   | **88.0±2.8**   | **98.4±2.2**     | **73.6±4.6**      |\n\n| Model               | put drawer | put money  | light bulb | slide block | place shape | stack blocks | stack cups | sweep dustpan | turn tap  | Average |\n|---------------------|------------|------------|-------------|--------------|--------------|---------------|-------------|----------------|------------|---------|\n| BridgeVLA w/o heat  | 0.0±0.0    | 58.7±22.7  | 2.7±2.3     | 64.0±0.0     | 4.0±4.0     | 0.0±0.0       | 0.0±0.0     | 32.0±4.0       | 40.0±10.6 | 31.4    |\n| BridgeVLA w pos     | 78.7±2.3   | 97.3±4.6   | 16.0±4.0    | 72.0±0.0     | 21.3±8.3    | 17.3±2.3      | 4.0±4.0     | 53.3±2.3       | 84.0±0.0  | 56.2    |\n| BridgeVLA           | **99.2±1.8**   | **99.2±1.8**   | **87.2±6.6**    | **96.0±2.8**     | **60.8±7.7**    | **76.8±8.7**      | **81.6±3.6**   | **87.2±1.8**       | **92.8±3.3**  | **88.2**    |\n\n---\n\n### **Concern3:  Lack of clarity in core components.**\n\nWe apologize for the confusion caused by our insufficient explanation. We will add a more detailed description to the Method section in the final version. As the full implementation is available on the anonymous project site and will be open-sourced upon acceptance, we believe this does not significantly affect reproducibility.\n\nBelow, we provide a more detailed explanation of the two components you mentioned:\n\n\n### 1) Convex Upsampling Module\n\nThis module performs adaptive, content-aware interpolation to upsample low-resolution features into high-resolution outputs. Unlike fixed methods (e.g., bilinear or nearest-neighbor), it learns pixel-wise interpolation weights, allowing for finer spatial detail recovery.\n\nIt consists of two parallel sub-networks:\n\n- **Output Network (`net_out`)**: A 3-layer convolutional network  \n  `in_dim → 2×in_dim → 2×in_dim → out_dim`,  \n  which generates low-resolution output features.\n\n- **Mask Network (`net_mask`)**: A 2-layer convolutional network that predicts interpolation weights with output dimension  \n  `(up_ratio² × up_kernel²)`. These weights are normalized using **softmax** to form a valid convex combination, ensuring stability and interpretability.\n\nThe pseudocode of the forward function is as follows:\n\n```python\ndef forward(self, x):\n    \"\"\"\n    :param x: (bs, in_dim, h, w)\n    :return: (bs, out_dim, h*up_ratio, w*up_ratio)\n    \"\"\"\n    bs, c, h, w = x.shape\n    assert c == self.in_dim, c\n    out_low = self.net_out(x)  # low resolution output\n\n    mask = self.mask_scale * self.net_mask(x)\n    mask = mask.view(bs, 1, self.up_kernel**2, self.up_ratio, self.up_ratio, h, w)\n    mask = torch.softmax(mask, dim=2)\n\n    out = F.unfold(\n        out_low,\n        kernel_size=[self.up_kernel, self.up_kernel],\n        padding=self.up_kernel // 2,\n    )\n    out = out.view(bs, self.out_dim, self.up_kernel**2, 1, 1, h, w)\n\n    out = torch.sum(out * mask, dim=2)\n    out = out.permute(0, 1, 4, 2, 5, 3)\n    out = out.reshape(bs, self.out_dim, h * self.up_ratio, w * self.up_ratio)\n\n    return out\n```\n\nThe entire process is **end-to-end differentiable**, making it suitable for joint training with the rest of the model. This module is adapted from the **RAFT** paper [1], and we encourage reviewers to refer to that work for further technical details.\n\n### 2）Loss function\n\n**For pre-training**, the heatmap prediction loss $\\mathcal{L}\\_{\\text{heat}} $is the only supervision signal. Therefore, the total pre-training loss is:  \n$$ \\mathcal{L}\\_{\\text{p}} = \\mathcal{L}\\_{\\text{heat}} $$\n\nSince the heatmap represents a probability distribution over 2D space, we use cross-entropy loss to measure the difference between the predicted and ground-truth heatmaps. Let $P$ be the ground-truth heatmap distribution, $Q$ the predicted distribution, $N$ the number of heatmaps, and $H$,$ W$ the height and width of each heatmap. Then:  \n$$ \\mathcal{L}\\_{\\text{heat}} = - \\sum\\_{n=1}^{N} \\sum\\_{i=1}^{H} \\sum\\_{j=1}^{W} P[i, j] \\cdot \\log Q[i, j] $$\n\n**For finetuning**, the total fine-tuning loss $\\mathcal{L}\\_f$ is composed of four parts:  \n$$ \\mathcal{L}\\_f = \\mathcal{L}\\_{\\text{heat}} + \\mathcal{L}\\_{\\text{rot}} + \\mathcal{L}\\_{\\text{grip}} + \\mathcal{L}\\_{\\text{col}} $$\n\n- **$\\mathcal{L}\\_{\\text{heat}}$**: Same as in pre-training. Since we compute the target position from the heatmap, this loss can supervise position prediction.\n\n- **$\\mathcal{L}\\_{\\text{rot}}$**: Supervises the predicted end-effector rotation. We discretize each Euler angle (x, y, z) into 72 bins and use cross-entropy loss. Let $x, y, z $denote the ground-truth distributions, and $\\hat{x}, \\hat{y}, \\hat{z} $ the predicted distributions. Then:  \n$$ \\mathcal{L}\\_{\\text{rot}} = - \\sum\\_{i=1}^{72} \\left( x[i] \\cdot \\log \\hat{x}[i] + y[i] \\cdot \\log \\hat{y}[i] + z[i] \\cdot \\log \\hat{z}[i] \\right) $$\n\n- **$\\mathcal{L}\\_{\\text{grip}}$**: Supervises the gripper open/close status (binary classification). Let $z\\_{\\text{grip}} \\in \\{0, 1\\}$ be the ground-truth label, and $\\hat{z}\\_{\\text{grip}} \\in [0, 1]$ the predicted probability of the gripper being open. Then:  \n$$ \\mathcal{L}\\_{\\text{grip}} = - \\left( z\\_{\\text{grip}} \\cdot \\log \\hat{z}\\_{\\text{grip}} + (1 - z\\_{\\text{grip}}) \\cdot \\log (1 - \\hat{z}\\_{\\text{grip}}) \\right) $$\n\n- **$\\mathcal{L}\\_{\\text{col}}$**: Supervises whether the predicted motion avoids collision (binary classification). Let $z\\_{\\text{col}} \\in \\{0, 1\\}$ be the ground-truth label, and $\\hat{z}\\_{\\text{col}} \\in [0, 1]$ the predicted probability of avoiding collision. Then:  \n$$ \\mathcal{L}\\_{\\text{col}} = - \\left( z\\_{\\text{col}} \\cdot \\log \\hat{z}\\_{\\text{col}} + (1 - z\\_{\\text{col}}) \\cdot \\log (1 - \\hat{z}\\_{\\text{col}}) \\right) $$\n\n[1] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020."}]

Author rebuttal:

Meta-review: BridgeVLA is a robot manipulation model that uses a 2D VLM to process the information from 3D inputs (point clouds) in order to gain significant sample efficiency across a variety of manipulation tasks. The experiments (in the paper itself and in the rebuttal) show the method's advantage over the baselines and a high absolute performance in very low-data regimes. 

The reviewers appreciated the data efficiency and generalization of the approach but pointed out the lack of heatmap-related ablations and comparisons to other VLAs as a weakness. The authors have addressed these by providing comparisons against SpatialVLA and pi_0 in the rebuttals. 

The metareviewer finds that the authors have factually mitigated the reviewers' concerns but that reviewer Prgj's and J4oj's final justifications and scores are misaligned with the content of their discussions with the authors. Reviewer Prgj is giving the paper a firm reject because of a non-learnable component that doesn't appear critically important to the paper's contribution and despite their other concerns getting addressed. Reviewer J4oj is giving the submission a weak reject despite declaring the intention to raise the score, due to the paper's evaluations focusing on low-data regimes and supposedly missing in-depth explanations of BridgeVLA's advantage over pi_0. In the meantime, the authors explicitly state that BridgeVLA specifically focuses on low-data regimes, something that pi_0 isn't explicitly targeting and isn't designed to be suited for.

Overall, despite some weaknesses, this works appears to be above the NeurIPS acceptance bar.
Decision: Accept (poster)