ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation
Authors: Jiawen Yu
Hairuo Liu
Qiaojun Yu
Jieji Ren
Ce Hao
Haitong Ding
Guangyu Huang
Guofan Huang
Yan Song
Panpan Cai
Wenqiang Zhang
Cewu Lu
Venue: NeurIPS 2025 poster ()
Keywords: VLA
Robotics
Contact-rich Manipulation
Robotics and Control
Primary area: reinforcement_learning
TL;DR: This work integrates the robot's perception of external forces as a novel modality into Vision-Language-Action models by leveraging a multimodal Mixture-of-Experts (MoE) architecture to capture subtle dynamic changes during interaction processes.

Abstract: Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations. However, they struggle with contact-rich tasks that require fine-grained control involving force, especially under visual occlusion or dynamic uncertainty. To address these limitations, we propose \textbf{ForceVLA}, a novel end-to-end manipulation framework that treats external force sensing as a first-class modality within VLA systems. ForceVLA introduces \textbf{FVLMoE}, a force-aware Mixture-of-Experts fusion module that dynamically integrates pretrained visual-language embeddings with real-time 6-axis force feedback during action decoding. This enables context-aware routing across modality-specific experts, enhancing the robot's ability to adapt to subtle contact dynamics. We also introduce \textbf{ForceVLA-Data}, a new dataset comprising synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. ForceVLA improves average task success by 23.2\% over strong $\pi_0$-based baselines, achieving up to 80\% success in tasks such as plug insertion. Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control. Code and data will be released at https://sites.google.com/view/forcevla2025/.

Force VLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation
Jiawen Yu Hairuo Liu Qiaojun Yu4.2t Jieji Ren Ce Hao Haitong Ding Guangyu Huang". Guofan Huang', Yan Song', Panpan Cai2.3, Wenqiang Zhang' Cewu Lu 2.3.8.
I Fudan University, SShnngaii Jiao Tong University, *Shanghai Innovation Institute, 4 Shanghai AI Lab, National University of Singapore, Shanghai University, Xi'an Jiaotong University, Noematrix Intelligence Equal contribution Corresponding authors
Abstract
Lision-Language-Aatii (VLA) models have advanced general-purpont robotic manipulation by leveraging pretrained visual and linguistic representations. How- ever, they struggle with contact-rich tasks that require fine-grained control involving force, especially under visual occlusion or dynamic uncertainty. To address these limitations, propose Force VLA, novel end-to-end manipulation framework that treats external force sensing as a first-class modality within VLA systems. VLA introduces FVLMoE. force-aware Mixture-of- -f-eeeett-s fusion module that dynamically integrates pretrained visual-language embeddings with real-time 6-axis force feedback during action decoding. This enables context-aware routing across modality modality-specific specific experts, enhancing the robot's ability to adapt to subtle contact dynamics. We also introduce ForceVLA--Date a new dataset comprising synchronized vision, proprioception, and force- torque signals across five contact- rich manipulation tasks. ForceVLA improves average task success by 23.2% over strong ro-based baselines, achieving up to 80% success in tasks such plug insertion. Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control. Code and data will released at webssite
Introduction
Robotic learning has advanced rapidly with the rise of embodied AI, driven by large-scale manip- ulation datasets and the emergence of foundation models [I] 2 3). These models exhibit strong
adaptability, enabling rapid generalization to novel tasks with minimal supervision [4] [5] 6). In parallel, Vision- Languuge Models (VLMs) have achieved remarkable generalization through large- scale multimodal alignment 7 g] learning transferable representations that support a wide range of downstream tasks.
Building on this progress, OpenVLA 9 introduced Vision- AssionLLanuugg--ctiin (VLA) models to bridge perception and control for real-world robotic manipulation. By leveraging VLM-based encoders, these models demonstrate strong performance semantic grounding, language following, and zero- shot generalization. [10] further enhances this framework using stronger VLM backbones [[] and flow-based action generation, showing that pretrained multimodal VLA models can acquire robust world priors and can be fine-tuned efficiently with only few demonstrations.
However, contact-rich manipulation demands more than semantic grounding and spatial planning- it is fundamentally driven by interaction forces [12 [3] Existing VLA models rely heavily on visual and linguistic cues, often overlooking force sensing, a modality critical for precise physical interaction. In contrast, humans naturally integrate tactile and proprioceptive feedback to adapt their manipulation
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

External Force
Instruction
TCP
TCP Pose
Pose
VLA Models without
Force VLA with
Wrong
Modify Insertion Posttion
Inital
Success
Fail
(a) VLA Models w/o force input
(b) ForceVLA with force modality input
Figure 1: Comparison between VLrceVLA and baselines without force input. Without force feedback, the policy fails correct pose errors and completes insertion incorrectly. In contrast, ForceVLA leverages external force signals to adjust insertion strategies dynamically, leading to successful execution despite initial misalignment.
strategies [4]. As a result, VLA models frequently struggle with tasks such as insertion, tool use, or assembly-especially under occlusion poor visual conditions-leading brittle behavior or task failure. Moreover, force requirements evolve across different task phases: delicate grasping, controlled insertion, and compliant surface contact-each requiring distinct forms of force modulation. Current methods lack mechanisms to perceive and adapt to these dynamic variations, limiting their ability reason over time about physical interactions.
To address these limitations, we introduce ForceVLA, a novel framework that augments VLA models with force-aware Mixture-of- (MoE) module, enabling effective reasoning and context-sensitive, force-informed action generation in contact-rich manipulation tasks, as illustrated in Figure VLAceVLA is grounded in the key insight that 6D external force sensed the robot's end-effector, should be treated as first-class modality and formally integrated into the action expert module enable phase-aware action generation based on force feedback during physical interaction. To realize this integration, Force incorporates a force-aware MoE module, named FVLMoE, designed to perform modality- and phase-aware fusion of linguitii representations with real-time force feedback from embodied interaction during action planning. Through gating mechanism, FVLMoE computes dynamic routing weights over expert subnetworks, each specialized for different modalities across task execution phases. By adaptively activating these experts based on high-level task instructions and low-level interaction feedback, VLA captures subtle yet critical, phase-dependent variations during physical interaction and generates precise, phase-aligned, and force-aware action chunking. To facilitate research in force-aware manipulation, we introduce Force LA-Data, carefully curated dataset comprising synchronized vision, proprioception, and 6-axis force-torque signals collected across five representative contact-rich manipulation tasks. This dataset addresses the scarcity of publicly available force force-annotated robotic manipulation data and provides a valuable resource for training and evaluating multi- colli- robotic learning systems. Our main contributions are:
We present a novel framework that integrates force, vision, language, and action for improved precision and stability on contact-rich manipulation tasks. Key to our approach is force-aware Exxture-ff-Expert--baseee
fusion module, which enables dynamic processing and deep integration of force, visual, and language features during action generation, significantly enhancing physical interaction capabilities in VLA systems.
We introduce a new dataset featuring synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. We also provide a complete data collection

---

pipeline unplline-including teleoperation tools and data converters and commit to open- all resources advance research force-aware robotic manipulation.
Through experiments on five challenging tasks, ForceVI.A achieves up to 80% task success and improves average performance by 23.2% over baselines, demonstrating strong generalization to novel objects, occlusions, and physical perturbations.
2 Related Works
Robotic VLA domain. Recent research Vision-Langunge-Action (VLA) models has focused on leveraging surge-scale multimodal pretraining to generalize robotic policies across tasks and 16 [7
20). These models typically map visual and language inputs to low-level control signals via end-to-end learning. Flow-based architectures such as To [T0 21 integrate pretrained vision-language encoders with fast action decoders to achieve high-frequency
embodiments [4] 6 15
18
19
outputs. Other works incorporate reasoning mechanisms [22 2 24 25], action space compression,
or 3D point cloud inputs [26] to improve instruction grounding and task execution. Diffusion-bast
models [5] 22 28 29
30 introduce stochastic generation for diverse, long- morghorizon behaviors, though
they often incur high training and inference costs. Despite these advances, most VLA approaches remain limited vision and language inputs, making them less effective in contact-rich or occluded manipulation scenarios where tactile feedback is critical.
Contact-rich manipulation domain. Traditional vision-only methods struggle with dynamic in- teractions requiring sine-graneed feedback. To address this, recent works integrate force sens- enabling improved motion stability and accuracy. Xie et al. [35 [3] provide foundational studies on the role of force feedback in robotic control. Tactile sensing has also emerged as a powerful modality: TLA (6) and Tac-Man
ing 31 32
33 34
(37 demonstrate enhanced performance in fine manipulation and articulation tasks. VTLA [8 explores sim-to-real transfer using fingertip tac- tile sensors with a new simulation dataset. In contrast to VTLA's simulation-first approach, our work adopts a woal-world-first paradigm with 6D force-torque sensing and introduces novel MoE architecture for online force modulation rather than focusing on sim-to-real transfer. Multimodal fusion methods [39 40) show promise in complex environments. FuSe [41 proposes finetuning recipe adapt existing generalist models incorporate new modalities. While FuSe focuses on
adaptation strategies, our contribution lies in designing a complete end-to-end policy architecture with deep integration of force feedback through an MoB-based routing mechanism. However, current approaches are often limited to static modality fusion and lack dynamic routing or unified modeling frameworks. Furthermore, few evaluate cross-task generalization in real-world contact-rich settings.
Contact-rich manipulation datasets. Large-scale datasets have been instrumental in advancing robotic learning. RH20T (42 provides valuable collection of diverse manipulation demonstrations, offering raw data with basic reading tools. Factory 43 introduces a powerful simulation toolkit for contact-rich interactions at scale, providing virtual assets and controllers for policy training in simulated environments. IndustReal (44) demonstrates an approach that explicitly avoids force-torque sensors, instead relying on alternative cues manage contact during manipulation. While these pioneering efforts have advanced the field in task diversity, simulation capabilities, and sensor-free approaches, they have limitations that our work addresses. Specifically, existing datasets often lack out-of the-the-ox usability for modern training frameworks, rely on simulation that introduces a sim-to- real gap, or deliberately exclude explicit force sensing. In contrast, our VLA- provides three unique contributions: (1) pre-processed data in the SOTA Lerobot format with conversion tools for plug-and-play compatibility, significantly reducing data-wrangling overhead; (2) a reality- first methodology with data collected entirely in the real world, bypassing the sim-to-real gap; and (3) deep integration of reghuresolution 6D force-torque sensor data, positioning our dataset as an essential resource for developing and benchmarking policies that explicitly leverage force feedback for dexterous manipulation.
MoE architecture-related work. cftture-of-Experee (MoE) architectures improve model scalability and efficiency by activating sparse expert subnetworks 45
46
47
48
49
Follow-up work [50
improves MoE training stability and task transferability. In the multimodal domain, LIMOE
[51
integrates sparse expert layers for joint vasion-language learning. Recent applications in roboties [52
29 adopt MoE layers within VLA models to enhance policy generalization and adaptability. However, these methods largely omit explicit modeling of the force/tactile modalities, and lack mechanisms for dynamically routing across multimodal signals in contact-intensive tasks.
3

---

3 Preliminary
Problem Formulation. Figure shows the set-
ting of robot manipulation tasks. The robot's observation at timestep t consists of base and hand visual inputs V2 and V2", the propriocep- tngle-timestep exter-
Langwage Instruction: "Peel Cucumber Skin."
tive state E IR,, and nal force-torque reading f. R which are collectively denoted as O (V. V/h St, Jt). Given a language instruction I., the objective is to learn an end-to-end policy x (A,,O,, L) that outputs low-level, executable action chunk A, {ae, d,-1, H++H-1 [[O] maximizing the likelihood of completing the contact-rich task, where is vector of TCP pose con- catenated with gripper width. TCP position is represented by Cartesian coordinates (I, y, 2) and orientation is represented by Euler angles (,, fz is the estimated external wrench applied on TCP at timestep i and expressed in world frame, which consists of R force and R moment: ft = (ftz. Says ftz, Maa, May. Mea
Visual Inpur (Hand)
External wrench
Visual Input (Basc)
TCP Puse
Figure 2: Robot manipulation tasks setting.
MoE Architecture. We select Mixture-of-
Experts (MoE) [47] as our fusion layer. The core idea is to distribute different modalities to larger set of smaller, specialized "expert" subnetworks, only fraction of which are activated for any given input token. An MoE layer typically comprises a set N expert networks, denoted as {E,) and gating network (also referred to router), denoted as G. This network takes input token x and dynamically determines which of the N experts should process it. In prevalent sparse MoE implementations, for input token v; the gating network G(z)) produces scores logits that are used to select a small subset of & experts (typically k = 1 or = 2, where & < N) from the total pool of N experts. The input token x is then routed only these It active experts. The outputs of these active experts, Eg (x),, are subsequently aggregated, commonly through a weighted sum where the weights (1) also derived from the gating network. The final output y(x) of the MoE layer can be expressed as: y(æ) = (o(to)((((e) (z) Ex(x), where TopK(G(z)) denotes the set of indices of the top-k experts selected by the gating network for input
PorceVLA
4.1 Overview of VLA
VLrceVIAA an end-to-end multimodal robotic policy designed for contact-rich manipulation. Its pipeline is illustrated in Figure Building upon the xo framework [IO], it integrates vision, language, proprioception, and 6-axis force feedback to generate actions through a conditional flow matching model [53 54] Visual inputs from multiple RGB cameras and task instructions are encoded by SigLIP- Sissed-based
35 lasion-language model (based on PaliGemma (TT] into contextual embeddings. These embeddings, combined with proprioceptive and force cues, condition an iterative denoising process that predicts the action trajectory.
FVLMoF is the core module enabling effective force integration. Force readings are linearly projected into dedicated tokens and fused with vision-language embeddings via Mixture-of Exprrtsss (MoE) module. Inspired by MoE's strength in multi-task and modality-specinf learning [56] 51, FVLMoE adaptively routes and processes multimodal inputs. Its output serves as a rich guidance signal for the flow model, allowing ForceVLA to handle subtle contact dynamics and visually ambiguous scenarios with greater precision and robustness.
4.2 FVLMoE Architecture
The FVLMoE module is specifically designed for the fusion of multimodal sensory information. Its design enables the model to integrate rich contextual understanding from vision and language with the immediate, fine-grained dynamics captured by force-torque sensors. This fusion is critical for enabling robust and adaptive behavior in contact-rich manipulation tasks. The architecture and operation of the FVLMoE can be detailed in the following stages:

---

Exccute
V-I. Pused Feature
Action Expect
stained
F-V-L Fused FeadFuaauu Feature
VTT
Layer
Norm
Export
Solf
Router
Lincur
FVL
feature
"Fick up the USB and into the socket."
Expert
Pereeive
New
Force
Updane
Layer
Project
FVLMoB
Jees
Visual Tokens
Languago Tokens
Force Tokens
Figure 3: Overview of our Lor model. Visual and language inputs are processed by a pre- trained VLM to form contextual embeddings. External force signals are projected and fused with VLM outputs via the FVLMoE module. The resulting multimodal features guide flow-based action head to generate contact-aware robot actions.
Input Mapping for Multiple Modalities. How to determine the optimal stage and method for a novel force modality incorporation poses a significant design challenge. Following extensive established approach where the force modality is introduced after the primary VLM has processed visual and linguistic inputs. Specifically, force features are fed as distinct inputs into the FVLMoE module, positioning it to function akin higher-level cortical association area responsible for integrating the VLM's pre-trained visual-linguistic representations with the newly introduced force tokens. This strategy contrasts with introducing force prior to, concurrently with, the VLM's initial fusion of visual and language modalities. The empirical justification for this architectural decision is elaborated in the Ablation Studies section (Section 5.4))
experimentation,
The FVLMoE module, line with this design choice, ingests sequence of token embeddings Ein formed by the concatenation of visual-linguistic features and a dedicated force token. The VL features, denoted as E E RN DDodall are outputs from the primary Vision- Dision-Language Model, encapsulating contextual understanding derived from processed image streams and textual instructions. Concurrently, the raw 6-axis force-torque sensor data, fraw E R transformed by a linear projection dr into force token embedding Er @r(fraw) E R.oodee The final input to the FVLMoE is thus the concatenated sequence E,n LE Eyl [(Nv++1)x Dmodell where the force token is appended to the visual- -enal-linguistic context for subsequent joint processing within the MoE architecture
Multimodal Routing and Fusion Computation. Once the combined multimodal sequence Ein e (N) is formed, undergoes hicrarchical processing within the FVLMoE module. Ei.. is passed through an encoder layer for shared refinement to facilitate holistic interaction among all constituent force, visual, and language tokens. This layer composed of a multi-head self- attention mechanism with Nheads attention heads and subsequent FFN, yielding Eene E R Pmodel, Subsequently, Eene channeled into a sparse Mixture-or of-Experts layer. This layer employs distinct expert networks, each realized as an independent MLP. A dynamic gating network determines routing, selecting the most appropriate single expert (top k = 1) for each token in Eenc based on learned dispatch weights. The output from the MoF. computation is then integrated back with the input to the MoE layer via a residual connection, yielding Efueed- The resulting sequence of fused multimodal features is passed through final linear projection layer to match the dimensionality of the action expert.
Injecting Fused Features into the Action Flow Head. The sequence of fused multimodal features produced by the FVLMoE module serves as a guidance signal for the action generation process, which is formulated flow-based denoising model. This guidance is materialized by first extracting a specific sub-sequence, GFvIMoE E IR XD.. comprising the final Haction tokens from EFVEMoE: Mart these tokens encapsulate the most pertinent fused guidance for each step in the -legth action plan. GrvIME is then combined via element- wise addition with Sullix RR Hamm xD, obtained from the primary VLM's processing of the current proprioceptive robot state st DR and the noisy
5

---

Action Pickop
Tanckoal Action Pris up
Desind
Toekol Action
Artic:
Pick
(a) USB
(b) Pump Bottle
Insert Plug
(d) Pool Cucumber
(c) Wipe Board
Figure 4: Overview of task setups used in evaluation. (a) Insert USB, (b) pump bottle, (c) insert plug. (d) peel cucumber, and (e) wipe board. These tasks span diverse contact dynamics and manipulation skills, from precise insertions to tool-mediated surface interactions.
action trajectory of IR HD at denoising step T, where D, and Da the dimensionalities of the state and action spaces, respectively. This additive injection mechanism ensures that the rich, contact-aware contextual understanding developed by the FVLMoE directly modulates and refines the generated action sequence at each step of the predicted trajectory.
4.3 Datasets
To train VLA, we curated new dataset specifically focused on contact-rich manipulation tasks, emphasizing the synchronized capture of visual, proprioceptive, and force-torque data. Existing datasets often lack the comprehensive force interactions or the diversity of contact-driven scenarios necessary to develop robust force-aware policies.
Our data collection was performed using Flexiv Rizon 7-DOF robotic arm equipped with a Dahuan adaptive gripper. Visual data was captured from two RGB-D cameras: one static third-person view (RealSense D435 at 1280x720, 30 FPS) and one wrist-mounted camera (RealSense D415 at 640x480, 30 FPS) providing egocentric perspectives. Data was collected via human teleoperation using a Quest3 VR interface with custom mappings to robot end-effector control. Five expert operators performed a total distinct contact-rich tasks: bottle pumping, plug insertion, USB drive insertion, whiteboard wiping, and cucumber peeling, described in Section 5.1 For each task, operators were instructed to complete the objective while ensuring diverse and successful interaction patterns. We varied object positions and orientations across demonstrations to enhance data diversity.
The resulting dataset, which we term PLAceVLA-Ded comprises a total of 244 trajectories, amounting to 140 thousand synchronized timesteps. All sensor streams were synchronized based on timestamps. Images were resized 480x640 pixels and normalized. Actions were represented as target TCP pose and gripper width. The ForceVLA-DD dataset, along with our data collection code and processing scripts, will be made publicly available at website to facilitate further research learning force-aware manipulation policies.
Experiments
This section presents comprehensive suite of real-world contact-rich manipulation experiments and analytical studies to empirically validate the Vor model. The evaluation is structured around four core research questions: (1) the overall effectiveness of Force VLA compared to baselines that incorporate force without our specialized fusion mechanism; (2) the model's ability to generalize across unseen object instances, environmental variations, and task conditions; (3) the efficacy of the proposed FVLMoE architecture in achieving optimal multimodal fusion for contact-rich manipulation; and (4) the ability of the modalities and learn meaningful routing behaviors across expert networks.
of-Exxerts module to appropriately process heterogeneous input
5.1 Experimental Setups
To evaluate the effectiveness of VLr, we conducted experiments on five diverse contact-rich manipulation tasks: Bottle Pumping, Plug Insertion, USB Drive Insertion, Whiteboard Wiping, and Cucumber Peeling, in Figure These tasks were chosen to assess fine-grained control, adaptability to varied initial conditions, and the utility of multimodal feedback, particularly force sensing. Each task introduces unique physical challenges: Bottle Pumping requires precise vertical pressing; Plug

---

106.
Shocos Rate (15
224
30
Insert
Ping
Pemp Bottle
Wips Board-1
Wipe Baard-2
Cucmmbr
Axurage
Main task success rates across different methods. Vor significantly outperforms all baselines on five contact-rich tasks. Incorporating external force feedback improves performance for To-base model, while our method achieves the highest average success rate. demonstrating robust performance under complex interaction dynamics. "Wipe Board-1" indicates the success rate of successfully performing the wiping motion, while "Wipe Board-2" refers to the success rate of completely erasing the markings.
Figure 5:
and USB Drive Insertions involve accurate alignment and force-controlled insertion; Whiteboard Wiping demands smooth trajectory control and surface contact; and Cucumber Peeling tests the ability to apply and maintain controlled force during continuous surface interaction.
We trained VLA using approximately 50 expert demonstrations per task. Evaluation was conducted over 20 trials each for the insertion and pumping tasks, 10 trials for the more time- consuming whiteboard task, and 15 trials for the cucumber peeling task, each involving peeling strokes. Success was defined by task-specific criteria such as complete insertion, effective wiping motion, or substantial cumulative peel coverage. These tasks were designed to rigorously probe VLrcsVLA's capacity to model and control complex, uncertain dynamics through the integration of vision, language, and force modalities.
Evaluation Metrics and Baselines. Model performance is primarily evaluated using the task success rate across all five challenging contact-rich manipulation tasks. For specific task like cucumber peeling, average peel length and minimum peeling times are also reported to provide a more nuanced assessment. To contextualize the performance of our proposed Vor model, we compare against several carefully selected baselines derived from the state-of-the-art To 10] architecture, which serves as our foundational model. The specific variants include To- 10 w/o (standard To without force input), To To-base [0] w/ F (To with force signals directly concatenated to state inputs), and corresponding fo-fast 25 configurations w/o and w/ F), representing potentially faster alternatives. The selection of bo-base 10) allows comparison with strong existing VLA method, while the 'inputForce' variants are crucial for demonstrating the efficacy of our FVLMoE fusion strategy over simpler foree integration approaches.
5.2 Main Results
Overall Performance. As demonstrated in Fig- 5 VorceVLA achieves an average success rate
Table I: Performance of cucumber peeling.
of 60.5% across all five tasks, significantly out- performing all baseline configurations. Compared to the standard moabase model without force feed- back -moe w/ F), which achieved average of 37.3%, VLA shows an improvement of 23.2%. This highlights the substantial benefit of incorporating and effectively processing multimodal information, including force. Table further highlights Force superior performance on the intricate cucumber peeling task. Our model excelled on both key metrics: it achieved the longest average peel length per stroke (14.12 cm 1), indicating better ability to execute -idelliy surface manipulation through stable tool orientation, adaptive contouring. and sustained surface contact compared to both -b-e w/ F (13.17 cm) and ba-e w/o F (10.27 cm). Concurrently, ForceVLA demonstrated superior overall efficiency by re- quiring the minimum number of strokes (7 4) to achieve a substantially peeled cucumber, significantly fewer than the 10 and 14 strokes needed by mo- w/ F and - w/o F, respectively. These combined results underscore Force VLA's proficiency in maintaining consistent, effective tool-surface
Avg. Peel Length (em)
Min. Strokes to Clean +
Model
-bsee [0] w/o F
10.27
14
-bse 10) w/ F
13.17
10
VLA (Ours)
14.12
7
7

---

Table 2: Success rates (%) of different models under various experimental conditions. Maximum values each column are highlighted in bold; second-best values are underlined.
Object
Object
Height
Visual Occlusion
Unstable Socket
Model
Average
Gen. 1
Gen. 2
Gen.
To-base
10
w/o F
48.00%
10.00%
66.67%
60.00%
10.00%
38.93%
To Tu-base
10
w/ F
32.00%
10.00%
77.78%
30.00%
10.00%
31.96%
To
Tu-fast
25
w/o F
80.00%
35.00%
88.89%
50.00%
10.00%
52.78%
To To-fast
25
w/ F
32.00%
5.00%
44.44%
50.00%
30.00% %
32.29%
VLr (Ours)
80.00%
40.00%
88.89%
90.00%
20.00%
63.78%
interaction and executing efficient, goal-directed motions in tasks demanding continuous and precise force modulation.
Effectiveness of Force Fusion via FVLMoE. Introducing raw force signals into Ta-base boosts performance from 37.3% 40.2%, confirming the utility of force feedback. However, Vor surpasses both with 60.5%, indicating that effective fusion-enabled by our FVI.MoE module- is essential for fully leveraging tactile information. This demonstrates that beyond the presence of force data, how it is integrated is critical to performance gains.
Selection of bose and To-fast model. For our foundational baseline, we evaluated -a-b and To-fast variants. The bo-base architecture demonstrated superior overall performance: To-base w/ F (40.2%) ) and -o-b w/o (37.3%) significantly outperformed Ty-fast w/ F (14.2%) and wo- w/o F (3100)). While fast variants exhibited a comparative advantage solely on the whiteboard wiping task -ese--ottenilllyyy due to a simpler action generation mechanism being more attuned such motions, the -a-t architecture's 's performance notably degraded (from 31.0% to 14.2%) when raw force input was directly added. We attribute this sensitivity to highly optimized and compact token space, which is likely disrupted by naively projected force tokens lacking corresponding large-scale pre-training. Conversely, Tg- ba-base modestly benefited from direct force input, with its larger representational capacity presumably allowing for partial utilization of these new sensory signals. Given its superior aggregate performance and more robust handling of naive force integration, To-base was selected as the primary baseline for developing and evaluating VLr.
5.3 Model Generalization
To evaluate Force generalization capabilities, we designed five experimental settings with increasing task variability and physical uncertainty, as illustrated in Figure 6 These settings include: (1) Object Gen. 1, which varies the bottle type in the bottle pumping task; (2) Object Gen. 2, which changes the plug type in the plug insertion task; (3) Height Gen., which adjusts the initial bottle height and measures success under torque limits; (4) Visual Occlusion, where parts of the plug and socket are obscured; and (5) Unstable Socket, introducing physical instability via clutter beneath the socket. These variations test both perceptual robustness and physical adaptability, with results summarized in Table 2
Across all settings, VLA exhibited superior generalization, particularly in scenarios requiring fine physical interaction. In Object Gen. 1, it achieved an 80.00% success rate, outperforming baselines that lacked force input or processed it naively. In the Height Gen. setting. Force VLA effectively scaled its interaction forces to variable depths, avoiding torque limit violations seen in other models. Furthermore, Vor maintained high success under visual degradation (90.00% in Visual Occlusion), reflecting its reliance on multimodal feedback beyond visual cues. Notably, while fa-f w/f performed best in the Unstable Socket task, this advantage stems from its frequent pausing wehavior-when the end-effector pauses while in contact with the plug, the socket is less likely wobble, effectively reducing the task difficulty. However, this stuttering limits its inference speed and reactiveness to real-time force changes, making it less competitive for general contact-rich manipulation tasks requiring prompt trajectory adjustments. These results underscore the critical role of the proposed FVLMoE architecture in intelligently integrating force information-aa just for sensing contact, but for modulating action in response to dynamic physical canditinns-enalliin more versatile and robust robotic manipulation.
5.4 Ablation Studies

---

(a) Object Gen.
(b) Object Gen.
(d) Visual Occlusion
(e) Unstable Socket
(c) Height Gen.
Variants of generalization settings used in our experiments. (a-b) Different object geome- tries; (c) variation in socket height; (d) partial visual occlusion; (e) unstable socket conditions. These scenarios evaluate robustness under diverse physical and perceptual perturbations.
Figure 6:
To validate the architectural design of For,eVLA, particularly the integration of force feedback, we conducted comprehensive ablation studies shown in Table [3 We compared early, late, and our proposed fusion strategy. Early fusion methods, such as "linear before VLM" and "MoE before VLM," which inject force data prior to the risual-language model (VLM), significantly degraded performance. Notably, the MoE-based early fusion failed entirely (0% success rate), highlighting that altering the input representations of a pretrained VLM disrupts its learned feature distributions and undermines its capacity to process lingul-ii signals effectively.
Late fusion strategies fared better. The "concatenate af- ter VLM" method, equivalent to a basic baseline where force features are appended at the decoding stage, im- proved success 60%- demonstrating the utility of force sensing, However, our proposed Force VLA architecture achieved markedly higher 80% success rate. By intro- ducing force features after the VLM's core encoding and using the FVLMoE module for adaptive fusion, Force VLA enables specialized routing and deeper multimodal inter- action. These results confirm two core design insights: force should introduced post-VLM VLM to preserve pretrained representations, and sophisticated fusion (via FVLMoEE is essential to fully leverage force in guiding contact-rich robotic behavior.
Table Ablation Results
Model
Success Rate
baseline
45%
linear before VLM
55%
MoE before VLM
concate after VLM
60%
ForceVLA (Ours)
80%
5.5 Visualization and Case Studies
Figure Ilustrates Force ability to adapt motion in response to contact feedhack during complex manipulation tasks. In the USB insertion task, when initial attempts failed due to misalignment, ForceVLA re-oriented or re-grasped the drive to achieve successful insertion -betavooss absent in baseline models, which repeated failed motions or applied excessive force. Similarly, in the "Unstable Socket" scenario (Figure 7p), FLA maintained compliant control as the socket shifted, dynamically adjusting the plug's 's pose to complete insertion, while baselines lost tracking and failed. These examples highlight key insight: simply adding force input does not ensure closed-loop adaptation. VL''s FVLMoE module enables deep fusion of force, vision, and language, supporting precise, context-aware control and robust generalization under dynamic physical conditions.
Conclusion
In conclusion, we propose VLA, framework that bridges the gap between high-level modality (vision/language) and low-level physical sensing (force) for contact-rich manipulation. At its core, ForceVLLA introduces FVLMoF, a Mixture-of--Eperts module that dynamically fuses visual, linguis- tic, and force modalities to enable fine-grained, context-aware control. Our experiments across five

---

(a) Insert USB
(b) Pump Bottle
(c) Insert Plug
Figure Trajectory visualizations across tasks and conditions. (a) USB insertion, (b) bottle pumping.
and (c) plug insertion under stable and unstable socket conditions. Each sequence illustrates how VLA adapts its actions in response to contact dynamics, retrying or adjusting pose when failures occur, ultimately achieving successful task completion.
challenging tasks show that VLA significantly outperforms strong To-bused baselines, achieving average success rate improvement of 23.2 23.2% and up to 80% success individual tasks. Ablation studies further validate the benefits of late-stage force fusion and expert routing. We also contribute Force VDaaaa new dataset for multimodal contact-rich manipulation. Through the co-design of our architectural approach and dataset, we demonstrate meaningful progress toward VLA systems that exhibit greater adaptive behavior and physical intelligence in manipulation scenarios.
Limitation. Firstly, ForceVLA currently utilizes estimated external wrench values. While this approach has proven effective, these estimations may not always capture the full precision afforded by direct high-fidelity measurements, particularly in scenarios demanding extreme haptic sensitivity. Potential enhancements include exploring the integration of superior sensors or advanced calibration techniques to further refine fine- grained control capabilities. Secondly, Force VLA's experimental validation was predominantly conducted robotic platforms with integrated, and typically high-cost, force-torque sensing, which can naturally limit broader accessibility. To promote wider practical deployment and help democratize force-aware manipulation research, we actively assessing LL'ss adaptability and performance on more common, lower-cost platforms equipped with external retrofitted force sensors. Thirdly, our work focuses exclusively on real-world data collection and evaluation to ensure findings are directly applicable to physical deployment scenarios, as mudti-body contact dynamics are central to our problem. While this real-world-firs approach addresses the sim-to-real gap, developing high-fidelity simulation environments for our diverse contact-rich tasks was beyond the scope of current resources. We recognize that large-scale training with simulated data represents promising avenue for expanding the range of tasks our model can address and view this as an important direction for future work.
Acknowledgment
This work was supported in part by the National Key R&D Program of China (Grant No. No.2024YFB4706600, National Natural Science Foundation of China (No.62576109, 62072112). National Natural Science Foundation of China under Grant 52505029, Natural Science Foun- dation of Shanghai under Grant 25ZR1401 1401 Science and Technology Major Project of Jiangsu Province (No.BG2024041), the Shanghai Commitee of Science and Technology, China(Grant 20.24511 103200) by the National Key Research and Development Project of China (N0.2022ZD01601002, Shanghai Artificial Intelligence Laboratory, XPLORER PRIZE grants.
10

---

References
[1] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: large-scale in-the-wild robot manipulation dataset. arXiv preprint M.Xi::2443.12944 2024.
[2] Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiimnt collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892-6903. IEEE, 2024.
[3] Homer Rich Walke, Kevin Black, Tony 7. Zhao, Quan Vuong, Chongyi Zheng, Philippe Flansen-Estruclt Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du et al. Bridgedata v2: A dataset for robot learning scale. Conference on Robot Learning, pages 1723- 1736.1 PMLR, 2023.
[4] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control scale. arXiv preprint 0rX1v:22/2.06817, 2022.
[5] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint 2rXiv:2405. 2024.
[6] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski. Tianli Ding. Danny Driess, Avinava Dubey, Chelsea Finn, et al. R1-2: lanoon-lnngagg-aatii models transfer web knowledge to robotic control. arXiv preprint 2rXiv:2307.15814 2023.
[7] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763. PmLR, 2021.
[8] Wenwen Zhuang, Xin Huang, Xiantao Zhang, and Jin Zeng. Math-puma: Progressive upward multimodal aligument to enhance mathematical reasoning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 26183-26191, 2025.
[9] Moo Jin Kim, Karl Pertsch. Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna. Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint (xxix:24060092440 2024.
[10] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. No: A vision langon--anggeeeeee flow model for general robot control. arXiv preprint arXiv:2410.22 2414,44444, 2024.
[11] Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang. Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, al. Paligemma: A versatile 3b vim for transfer. arXiv preprint arXiv:2407.0777 2024.
[12] Xiang Zhang, Changhao Wang. Lingfeng Sun. Zheng Wu, Xinghao Zhu, and Masayoshi Tomizuka. Efficient sim-to-real transfer of contact-rich manipulation skills with online admittance residual learning. In Conference on Robot Learning, pages 1621-1639, PMLR, 2023.
[13] Michael Noseworthy, Bingjie Tang, Bowen Wen, Ankur Handa, Chad Kessens, Nicholas Roy, Dieter Fox, Fabio Ramos, Yashraj Narang, and Iretiayo Akinola. Forge: Force-guided exploration for robust contact-rich manipulation under uncertainty. IEEE Robotics and Automation Letters, 2025.
[14] Sung Soo Kim. Manuel Ramirzz Pramodsingh Thakur, and Steven S Hsiao. Multimodal interactions between proprioceptive and cutancous signals primary somatosensory cortex. Neuron, 86(2):555-566, 2015.
[15] Moo Jin Kim, Chelsea Finn, and Perey Liang. Fine-tuning vision-languagc-acttion models: Optimizing speed and success. arXiv preprint arXiv:2502. 19645, 2025.
[16] Wei Zhao, Pengxiang Ding, Min Zhang. Zhefei Gong, Shuanghao Bai, Han Zhao, and Donglin Wang. Vlas: Vision acctin model with speech instructions for customized robot manipulation. arXiv preprint arXiv 2025.

---

[17] Jiaming Liu, Mengzhen Liu, Zhenyu Wang. Lily Lee, Kaichen Zhou, Pengju An, Senqiao Yang, Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Multimodal state space model for efficient robot reasoning and manipulation. arXiv preprint urXiv:2406.04339, 2024.
[18] Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, and Linxi Fan. Vima: General rohot manipulation with multimodal prompts. arXiv preprint Xiv: 10.03094, 2(3)6, 2022.
[19] Johan Bjorck, Fernando Castañeda, Nikita Chemiadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang. Dieter Fox, Fengyuan Hu. Spencer Huang. et GrOOt nl: An open foundation model for generalist humanoid robots. arXiv preprint ar0iv: 14734, 2025.
[20] Cumxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, et al. Interleave vla: Enhancing robot manipulation with interleaved image-text instructions. arXiv preprint arXiv:2505.022 0212, 2025.
[21] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. np.5: a lision-laguuage-ctinn model with open-world generalization. arXiv preprint arXiv:2504. 16054, 2025.
[22] Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of-thought reasoning for vision-languuge-cction models. arXiv preprint arXiv:2503.22000 2025.
[23] Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. Dexvla: Vision- language model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.08855 2025.
[24] Qingwen Bu, Hongyang Li, Li Chen, Jisong Cai, Jia Zeng, Heming Cui, Maoqing Yao, and Yu Qiao. Towards synergistic, generalized, and efficient dual-system for robotic manipulation. arXiv preprint arXiv:2410.08001, 2024.
[25] Karl Pertsch, Kyle Stachowicz. Brian Ichter. Danny Driess, Suraj Nair. Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models, arXiv preprint arXiv:2501. 099477, 2025.
[26] Chengmeng Li, Junjie Wen, Yan Peng, Yaxin Peng, Feifei Feng, and Yichen Zhu. Pointvla: Injecting the 3d world into lasion-langaag--cction models. arXiv preprint arXiv:2503.00 2025.
[27] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241223668, 2023.
[28] Yanjiang Guo, Yucheng Hu, Jianke Zhang, Yen-Jen Wang, Xiaoyu Chen, Chaochao Lu, and Jianyu Chen. Prediction with action: Visual policy learning via joint denoising process. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[29] Zhongyi Zhou, Yichen Zhu, Minjic Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng, Ran Cheng, Yaxin Peng, Chaomin Shen, et al. Chatvla: Unified multimodal understanding and robot control with vision-language-actoon model. arXiv preprint arXiv:2502. 10.02.1.42000 14420, 2025.
[30] Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang. Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, et al. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-actinn model. arXiv preprint arXiv:2503. 10631, 2025.
[31] Yansong Wu, Zongxie Chen, Fan Wu, Lingyun Chen, Liding Zhang, Zhenshan Bing, Abdalla Swikir, Sami Haddadin, and Alois Knoll. Tacdiffusion: Force-domain diffusion policy for precise tactile manipulation. arXiv preprint arXiv:2409, 10447 2024.
[32] Yifan Hou, Zeyi Liu, Cheng Chi, Eric Cousineau, Naveen Kuppuswamy, Siyuan Feng, Benjamin Burchfiel, and Shuran Song. Adaptive compliance policy: Learning approximate compliance for diffusion guided control. arXiv preprint arXiv:2410.09309, 2024.
[33] Wenhai Liu, Junbo Wang, Yiming Wang, Weiming Wang, and Cewu Lu. Forcemimic: Force-centric imitation learning with force-motion capture system for contact-rich manipulation. arXiv preprint arXiv:2410.07554. 2024.
[34] Zihao He, Hongje Fang, Jingjing Chen, Hao-Shu Fang, and Cewu Lu Foar. Force- lorce-aware reactive policy for contact-rich robotic manipulation. arXiv preprint arXi::2411.15533 2024.
12

---

[35] William Xie and Nikolaus Correll. Towards forceful robotic foundation models: literature survey. arXiv preprint arXiv:2504.1 11827, 2025.
[36] Peng Hao, Chaofan Zhang, Dingzhe Li, Xiaoge Cao. Xiaoshuai Hao, Shaowei Cui, and Shuo Wang. Tla: Tactile-language-acton model for contact-rich manipulation. arXiv preprint arXiv:2503. 2025.
[37] Zihang Zhao, Yuyang Li. Wanlin Li, Zhenghao Qi, Lecheng Ruan, Yixin Zhu, and Kaspar Althoefer. Tac-man: Tactile-informed prior-free manipulation of articulated objects. IEEE Transactions on Robotics, 2024.
[38] Chaofan Zhang, Peng Hao. Xiaoge Cao, Xiaoshuai Hao, Shaowei Cui, and Shuo Wang. Vtla: Vision- mecill-language-actiooo model with preference learning for insertion manipulation. arXiv preprint arXiv:2505.098 09577, 2025.
[39] Yiyang Ling, Karan Owalekar, Oluwatobiloba Adesanya, Erdem Bryık, and Daniel Seita. Impact: Intel- ligent motion planning with acceptable contact trajectories via vision-language models. arXiv preprint arXiv:2503. 101 10, 2025.
[40] Huaijiang Zhu, Tong Zhao, Xinpei Ni, Jiuguang Wang, Kuan Fang, Ludovic Righetti, and Tao Pang- Should we learn contact-rich manipulation policies from sampling-basse planners? IEEE Robotics and Automation Letters, 2025.
[41] Joshua Jones, Oier Mees, Carmelo Sterrazza, Kyle Stachowicz, Pieter Abbeel, and Sergey Levine. Beyond sight: Finetuning generalist robot policies with heterogeneous sensors via language grounding. arXiv preprint ar arXi::2001.066 04693, 2025.
[42] Hao-Shu Fang. Hongje Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu, and Cewu Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. arXiv preprint arXiv:2307 005995 2023.
[43] Yashraj Narang, Kier Storey, Iretiayo Akinola, Miles Macklin, Philipp Reist, Lukasz Wawrzyniak, Yunrong Guo, Adam Moravanszky, Gavriel State, Michelle Lu, al. Factory: Fast contact for robotic assembly. arXiv preprint Siti::220000332222 2022.
[44] Bingjie Tang, Michael A Lin, Iretiayo Akinola, Ankur Handa, Gaurav Sukhatme, Fabio Ramos, Dieter Fox, and Yashraj Narang. Industreal: Transferring contact-rich assembly tasks from simulation to reality. arXiv preprint arXiv:2305.1 /71 2023.
[45] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outragcously large neural networks: The sparsely-gated mixture-of offpeet- layer. arXiv preprint 17Xiv:1701.06536 2017.
[46] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(12)):1-3, 2022.
[47] Dmitry Lepikhin, HyoukJoong Lec, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang. Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Sealing giant models with conditional computation and automatic sharding. arXiv preprint arxiv:2006. 16668, 2020.
[48] Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. Glam: Efficient scaling of language models with mixture- of-experss. In International conference on machine learning, pages 5547-5569, PMLR, 2022.
[49] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Su- sano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts, Advances in Neural Information Processing Systems, 34:8583- 8595, 2021.
[50] Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. St-mae: Designing stable and transferable sparse expert models. arXiv preprint arXiv:2202.08996 2022.
[51] Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby. Multimodal contrastive learning with limoc: the language-image mixture of experts. Advances in Neural Information Processing Systems, 35:9564-9577 9976, 2022.
[52] Han Zhao, Wenxuan Song, Donglin Wang. Xinyang Tong. Pengxiang Ding, Xuelian Cheng, and Zongyuan Ge. More: Unlocking scalability in reinforcement learning for quadruped vision-language language- action models. arXiv preprint arXiv:2503.0m
13

---

[53] Qiang Liu. Rectified flow: A marginal preserving approach optimal transport. arXiv preprint
S.Xi:2200911557,, 2022.
[54] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXi::2210.2747. 2022.
[55] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings the IEEE/CVF international conference computer vision, pages 11975 11986, 2023.
[56] Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-expers.. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1930-1939, 2018
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiaton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The answer NA means that the abstract and introduction do not include the claims made in the paper
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification:
The paper clearly states the limitation of this work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
No theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The paper includes details for experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
This paper includes details for the experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ 7/nips. /ttppp.cc/pbllc/guideeee http://oeeuuiiisssoooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: This paper includes details for the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The results compare with baselines and ablation studies.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
This paper provides the training details and required computation resources.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neur/ /ttnghttttr///prrro....ooo/a///t
Answer: [Yes]
Justification: This paper follows the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper is not related societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: This paper does not have the risk of misuse language models.
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: This paper is not related to assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: This paper includes new datasets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper is not related to human subjects.
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our L.M policy (tteee///earrpppcccc///ffrerrrrp for what should or should not be described.
21

---

A
Data Collection System
Wrist Camera
Third-person
Camera
Operator
Teleoperating
(a) Data Collection with Teleoperation Setup
(b) Teleoperation Demonstration
Figure 8: Our data collection system setup.
Our data collection system is depicted in Figure Sta). The setup features a robotic arm equipped with both wrist-mounted camera and static third-person view camera capture diverse visual perspectives. An operator, wearing a Quest 3 headset and using hand-held controllers, teleoperates the robot arm. A nearby computer runs the necessary software for data acquisition, which includes programs for interfacing with the robot, synchronizing sensor streams, and managing communication with the VR teleoperation hardware. Figure (b)) illustrates the operator teleoperating the robotic arm to collect demonstration data for the wipe board task.
B Training Details
The models were mainly trained on compute nodes equipped with 8x NVIDIA RTX 4090 GPUs (24 GB VRAM each), 64 physical CPU cores, and 251 GB system RAM, using Adam optimization (3 0.9, 32 = 0.95) with a peak learning rate of 2.5 x 10 decaying to 2.5 x 10-6 over 30,000 steps. Multi-task training utilized data parallelism across 2 GPUs (global batch size 16, effective 2048 via gradient accumulation) as additional GPUs provided diminishing returns due to communication overhead, completing 30,000 steps in 12 hours, while single-task training used 1 GPU for 10,000 steps (~9 hours), both employing bolat1 precision with gradient clipping [| = 1.0).
The dimensionalities and key parameters of Force VLA's core processing modules: Input Projections, the FVLMoE block, and the Action Output Head are detailed in Table
Table Focused view of Force VLA's key architectural components: Input Projections, FVLMoE, and Action Output. Dimensions are indicative (e.g., DVLM. Daa for VLM and Action Expert).
Layer
Key Parameters/ Dimensions
Input Projections
Force Projection
Linear; Input: 6 (F/T), Output: DVLM = 2048
State Projection
Linear; Input: Danee 32, Output: D = 1024
Action Projection
Linear; Input: Daction 32, Output: Daa a 1024
Action-Time MLP
2-layer MLP: Input: x Duce. Hidden/Outpur: Duas: Swish activation
FVLMoE Module
Input
Concatenation: Ny1. x DVLM (V-L features) & 1 x DVLM (Force token) Transformer Encoder Block; Dmode 2048, Nu 8, Dn 256; MLP (cxpansion factor
Pre-MoE Encoder
MoE Layer
Sparse MoE; = experts (MLPs: Dmasel + Dmods). Top = 1; Router: Dmootel + E Linear; Input: Dmulel 2018, Output: DaLe 1021
Output Projection
Action Output Head
Action Output Projection
I Linear; Input: Dace = 1024, Output: Dacion 32
22

---

C
Router Analysis
Wipe
Fec
Figure 9: Open-loop evaluation of expert load across different task completion percentages for various tasks: (a) Pump Bottle, (b) Insert Plug, (c) Wipe Board, (d) Peel Cucumber, and (e) Insert USB. Each subplot represents the average expert load (vertical axis) as function of the task completion percentage (horizontal axis) over the episodes
test dataset.
To analyze routing dynamics, first measured the probability distribution over expert selections for each token as was processed by the router in the MoE module. For variable-length episodes, applied percentile-bai
normalization: each task (~10 episodes) was processed by segmenting every episode's s token sequence into 100 temporally equidistant intervals, computing the mean top- probability per segment, and then averaging these means across episodes. This ensured cross-episode comparability while preserving temporal routing dynamics.
As shown in Figure 9 different tasks exhibit distinct expert utilization patterns. Some tasks (c.g.. insert plug, peel cucumber) show clear temporal specialization, where certain experts dominate specific phases of the task. Others (e.g., wipe board) demonstrate single expert throughout execution. These trends suggest that the router learns to allocate computation dynamically across experts based on spec-fpe semantics and temporal structure.
consistent preference for
What's more, we found that Expert dominates nearly half of the tokens across multiple tasks. This persistent activation suggests that Expert O may function as a general-purpose expert, responsible the fusion of multiple modalities routine control primitives that are shared across tasks. Its broad vementement contrasts with the more selective, phase-specific activation of Expert or Expert reinforcing the hypothesis of functional specialization among experts. Such asymmetry in routing frequency reflects not only temporal semantic variance within tasks but also architectural bias toward certain experts, potentially shaped during training.
D Multi-task Evaluation
Table 5: Multi-task joint training success rates (%). Vor (Ours) demonstrates superior average performance and excels or matches the best performance in all individual tasks. Best performance(S) in each column are in bold.
Pump Bottle
Insert Plug
Insert USB
Wipe Board
Average SR
Model
No fast w/o
0.0%
0.0%
0.0%
0.0%
0.0%
To fast w/ F
0.0%
0.0%
0.0%
0.0%
0.0%
To base w/o
20.0%
0.0%
0.0%
0.0%
5.0%
To-hase w/F F
50.0%
100.0 %
10.0%
10.0%
42.5%
VLA (Ours)
80.0%
100.0% %
10.0%
80.0%
67.5%
23

---

The results of joint multi-task learning are detailed in Table Notably, both Tu-fast variants (0% average success rate) failed acquire skills in this setting, indicating their limited capacity for diverse, concurrent learning. The To-base w/o F model also performed poorly (5% average success rate), managing only 20% success on a single task. Adding direct force input (ma-base w/ F) improved the average performance to 42.5%, primarily due to its success in the insert plug task. Our ForceVLA model demonstrated superior multi-task capabilities, achieving the highest average success rate of 67.5%. It obtained 80% success in both pump bottle and wipe board tasks, and matched top performance in insert plug (100%) and insert usb (10%). ). This robust performance across multiple distinet tasks indicates VLA's capacity for concurrent skill learning, proficient instruction following for varied goals, and highlights the role of its FVLMoE architecture in utilizing multimodal cues, particularly force, within shared policy.
E Detailed Breakdown of Failure Modes
Plug Insertion:
w/o force models generally exhibit cruder behavior during insertion, unable adjust their strategy based on interaction state changes. w/ force models can generally perceive changes in external forces but are not flexible enough in adjusting their trajectory. ForceVLA can better maintain continuous contact, follow the plug's 's surface, make fine adjustments to the force angle and duration within a more precise range, and terminate the action more promptly upon full insertion.
The overall success rate for this task is low. A common problem across all five models is the difficulty in aligning with the USB port, which we attribute to insufficient visual clarity and the backbone model's 's inability to process fine-grained visual information. However, Force VLA's success rate is slightly higher because it exhibits clear autonomous adjustment or re-attempt behaviors upon feeling external force from contact with the socket.
USB Insertion:
Simply pressing a seen bottle could be completed with 100% success by the pi0-base models and VLA. Therefore, during testing for this task, we introduced additional visual occlusions and background changes. Most of the failure modes occurred under these specific variations. w/o force models often missed the pump or did not press fully. ForceVLA was more robust but sometimes pressed off-center.
Bottle Pumping:
w/o force models were significantly more prone to breaking the peel mid-way, unable to peel continuously from end to end, and sometimes peeled too deeply, indicating a lack of control over the peeling force. w/ force models peeled more stably with a wider peel, but still had issues with breaking the peel and not following the cucumber's s curvature well. VLA could overcome these issues.
Cucumber Peeling:
Board Wiping: A common problem for all models was the inability to pick up the eraser, possibly because its placement was far from the base camera, resulting in low resolution, and its color was similar to the black table, making precise localization difficult. In the remaining trials, if small part of the writing wasn't erased, we classify as a failure. Additionally, if model didn't 't stop after 5 minutes, we counted it as a failure. The w/o force models started wiping in the air or pressed too hard, leaving scratches, due to their inability to perceive force. Force made closer contact with the board and applied more appropriate wiping force than other models.
F Ablation Study: Force Input Masking
To directly assess the value of the force modality within our MoE architecture, we conducted an ablation study where the force imput to the ForceVLA model was masked during inference. Table 6 presents the quantitative results comparing ForceVLA with and without force inputs across four contact-rich manipulation tasks. Note that the Peel Cucumber task was not included this ablation due to time constraints.
As shown in Table removing force inputs leads to substantial performance degradation in most tasks, with particularly severe drops in Plug Insertion (60%). Pump Bottle (37%), and USB Insertion (25%). The Wipe Board task shows minimal degradation (0.3%), likely due to its relatively simpler contact dynamics.
24

---

Table 6: Success rates comparison between ForceVLA and ForceVLA without force inputs. The drop column indicates the absolute percentage decrease in success rate when force information removed.
Task
Force (%)
VorceVLA w/o Force (%)
Drop (%)
Plug Insertion
80.0
20.0
60.0
USB Insertion
25.0
0.0
25.0
Pump Bottle
67.0
30.0
37.0
Wipe Board
27.0
26.7
0.3
E1 Failure Mode Analysis
Through detailed analysis of the failed attempts, identified several characteristic failure modes when force information unavailable:
The end-effector could only push a short distance in the insertion direction, with insufficient trajectory curvature to properly insert the plug into the socket. Alternatively, the end- effector would move to the plug's surface and perform an arcing push motion near the contact point without establishing actual contact, resembling an overfitted insertion trajectory.
Plug Insertion:
The gripper's closing width was excessively large, preventing it from securely grasping the USB drive. This behavior suggests overfitting to the gripper width values observed during training, resulting in an inability to achieve firm grasp.
USB Insertion:
Pump Bottle: The policy repeatedly applied excessive force, causing the gripper to bend. Addi- tionally, off-center pressing caused the bottle to spring away from the gripper, indicating poor force regulation.
Wipe Board:
The number of wiping attempts was significantly reduced, resulting in most cases where the board surface not adequately cleaned.
F.2 Discussion
We hypothesize that these failure modes arise from the following mechanism: Without force input, the FVLMoE modulc's 's increased parameter count makes the model more susceptible to overfitting. During inference, the policy exhibits stereotypical behaviors that are misguided by vision alone, reflecting an overfit to the action distribution of the training data rather than reactive responses to actual contact conditions.
Conversely, the fact that the full ForceVLA model with the additional force modality does not suffer from this overfitting strongly indicates that our architectural design effectively leverages force signals to regularize the policy. This regularization enables the model to achieve robust, reactive behaviors instead of merely memorizing training trajectories, thereby validating the importance of force feedback in our multimodal MoE framework.
G Real-world Experiments Visualization
In this section, we present key frames from real-world experiment videos. Each visualization contrasts failure cases of baseline models with successful task completions by our Force model under similar conditions.
25

---

Pump Bottle
fast- w/o
fast- w/ F
base- w/O F
base- w/o F
Force VLA
Force VLA
Force VLA
Figure 10: Key frames from Pump Bottle task videos.
26

---

Insert USB
fast-
w/o F
fast- w/ F
basc- w F
basc- w/ F
basc-
w/o F
Force VLA
Figure 11: Key frames from Insert USB task videos.
Wipe Board
fast- w F
fast- w/ F
base-
w F
base-
w/ F
Force VLA
Figure 12: Key frames from Wipe Board task videos.
27

---

Insert Plug
fast- w F
fast-
w F
base-
w/o
base-
w/
Force
VLA
Figure 13: Key frames from Insert Plug task videos.
Insert Plug Gen1
fast- w F
fast- W/
base-
w/o F
base-
w/ F
Force
VLA
Figure 14: Key frames from Insert Plug Generalization task I videos.
28

---

Insert Plug Gen2
fast- W F
fast- w/ F
base-
w/ F
base-
w F
Force VLA
Figure 15: Key frames from Insert Plug Generalization task 2 videos.
Insert Plug Gen3
fast-
w F
fast- w/ F
base-
w/o F
base-
w
Force
VLA
Figure 16: Key frames from Insert Plug Generalization task 3 videos.
29

---

Insert Plug Oce1
fast- w F
fast- w/ F
base-
w/O
base-
w F
Force VLA
Figure 17: Key frames from Insert Plug Occlusion task I videos.
Insert Plug Oce2
fast-
w F
fast-
w/ F
base-
w
base-
w
Force
VLA
Figure 18: Key frames from Insert Plug Occlusion task 2 videos.
30

---

Insert Plug Unstable
fast- w F
fast- w/ F
base-
w F
base-
w F
Force VLA
Figure 19: Key frames from Insert Plug Unstable task videos.
31


Reviews:
[{"review_id": "hkKtPoHdn7", "reviewer": "Reviewer_1t4m", "summary": "This paper presents ForceVLA, a method for enhancing vision-language-action (VLA) models with force information using a mixture-of-experts (MoE) policy architecture. The proposed approach is evaluated on a real-world robot across several manipulation tasks. While the idea of integrating force signals into VLA frameworks is appealing and timely, the paper leaves several architectural and conceptual questions unresolved.", "questions": "1. Can you clarify whether each expert in the MoE is intended to specialize in different modalities? If so, how is this encouraged during training?\n2. What exactly is included in a \"force observation\"? Is it a single reading, or a sequence?\n3. Given that the teleoperator receives no force feedback, how are successful force-sensitive demonstrations collected?\n4. Why does pi0 perform worse with added force information in Table 2?\n5. What are the typical failure modes that explain the relatively low success rates?\n6. Can you include a comparison to a MoE model trained without access to force inputs?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n* Exploring force-enhanced VLAs is a timely and relevant problem.\n* Real-robot experiments are conducted on diverse tasks with promising results.\n* The mixture-of-experts formulation show potential to leverage multimodal signals.\n\n**Weaknesses:**\n\n* The related work omits several recent efforts combining vision, language, and tactile/force signals (e.g., VTLA, FuSe). A broader and more up-to-date literature review would better situate the paper.\n* From the introduction, one would expect experts in the MoE to specialize in different modalities (e.g., one for vision, one for force). However, the architecture does not enforce this: modalities are simply concatenated, and all experts receive the same fused input.\n* It is unclear what exactly comprises a \"force observation\"—e.g., Fig. 3 suggests a snippet of force trajectories rather than a single reading, but this is not explained clearly.\n* There is no force feedback during teleoperation, which raises doubts about whether the demonstrations are truly reactive to force or just to delayed visual cues.\n* Results are presented as tables of numbers with little analysis. Success rates remain low. Why? What are the main failure modes?\n* A comparison with MoE models trained without force input is missing, which would be important to isolate the value of the force modality.\n* In Table 2, performance of pi0 degrades when force is added (without MoE). Why does force hurt performance in this case?", "was_revised": true, "final_justification": "The authors have addressed most of my concerns. The justification about low success rates in some tasks is not fully convincing (e.g., humans can perform force-driven USB insertion with minimal visual feedback), but overall, this can still be a helpful contribution to the community.", "rebuttal": "We sincerely thank Reviewer 1t4m for the thoughtful and detailed feedback. Your comments on architectural clarity, experimental analysis, and missing comparisons are invaluable. They have helped us identify key areas for improvement, and we have addressed each point below.\n***\n### 1. On Architectural & Conceptual Clarity\n**Regarding MoE Specialization (Weakness 2, Question 1):**\nWhile it is challenging to definitively assign a single modality to each expert, our analysis reveals clear specialization patterns based on task and execution phase. As detailed in Section C of our supplementary material, we observe that the router learns to dynamically allocate different experts for different stages of a task. For example, Figure 9 shows that the expert load varies across different task completion percentages. This indicates that the router has learned a strategy to leverage different experts based on the evolving context of the task, rather than a static modality assignment.\n\n**Regarding the Definition of \"Force Observation\" (Weakness 3, Question 2):**\nThank you for this question highlighting a point of ambiguity. To clarify, a \"force observation\" is a reading from a single frame/timestep, not a sequence over time. Specifically, at each observation step, this single reading is a 6-dimensional vector representing the estimated external wrench applied on the TCP, expressed in the world frame ($^0F_{ext}\\in \\mathbb{R}^{6\\times 1}$). This vector consists of a $\\mathbb{R}^{3\\times1}$ force component and a $\\mathbb{R}^{3\\times1}$ moment component: $[f_x,f_y,f_z,m_x,m_y,m_z]^T$. The units are Newtons [N] for force and Newton-meters [Nm] for moments. We will revise the paper to make this distinction between a single-step reading and a time-sequence explicit to avoid any confusion.\n\n**Regarding Force-Sensitive Data Collection (Weakness 4, Question 3):**\nAlthough the human operator does not receive direct haptic feedback, they are provided with real-time visual feedback of the forces within their Quest3 VR headset. The 6D external wrench readings from the robot's sensor are streamed to the headset and visualized as a dynamic force vector originating from the robot's TCP. This vector continuously changes its length and orientation based on the measured force, allowing the operator to see and react to contact forces during teleoperation, thus enabling the collection of force-reactive demonstrations.We acknowledge that teleoperation systems with direct haptic force feedback are now available. Due to hardware constraints in our current setup, we did not integrate this capability. We agree that this is a valuable direction and will certainly consider incorporating direct haptic feedback in our future work to potentially enhance the quality and reactivity of the demonstrations.\n***\n### 2. On Analysis of Experimental Results\n\n**Regarding Typical Failure Modes and Low Success Rates (Weakness 5, Question 5):**\nWe have compiled a detailed breakdown of the failure modes for each task and model:\n- Plug Insertion: w/o force models generally exhibit cruder behavior during insertion, unable to adjust their strategy based on interaction state changes. w/ force models can generally perceive changes in external forces but are not flexible enough in adjusting their trajectory. ForceVLA can better maintain continuous contact, follow the plug's surface, make fine adjustments to the force angle and duration within a more precise range, and terminate the action more promptly upon full insertion.\n- USB Insertion: The overall success rate for this task is low. A common problem across all five models is the difficulty in aligning with the USB port, which we attribute to insufficient visual clarity and the backbone model's inability to process fine-grained visual information. However, ForceVLA's success rate is slightly higher because it exhibits clear autonomous adjustment or re-attempt behaviors upon feeling external force from contact with the socket.\n- Bottle Pumping: Simply pressing a seen bottle could be completed with 100% success by the pi0-base models and ForceVLA. Therefore, during testing for this task, we introduced additional visual occlusions and background changes. Most of the failure modes occurred under these specific variations. w/o force models often missed the pump or did not press it fully. ForceVLA was more robust but sometimes pressed off-center.\n- Cucumber Peeling: w/o force models were significantly more prone to breaking the peel mid-way, unable to peel continuously from end to end, and sometimes peeled too deeply, indicating a lack of control over the peeling force. w/ force models peeled more stably with a wider peel, but still had issues with breaking the peel and not following the cucumber's curvature well. ForceVLA could overcome these issues.\n- Board Wiping: A common problem for all models was the inability to pick up the eraser, possibly because its placement was far from the base camera, resulting in low resolution, and its color was similar to the black table, making precise localization difficult. In the remaining trials, if a small part of the writing wasn't erased, we classify as a failure. Additionally, if a model didn't stop after 5 minutes, we counted it as a failure. The w/o force models started wiping in the air or pressed too hard, leaving scratches, due to their inability to perceive force. ForceVLA made closer contact with the board and applied a more appropriate wiping force than other models.\n\nWe will integrate this detailed failure analysis into the appendix to provide a much richer context for our quantitative results.\n\n**Regarding π₀ Performance with Force Input (Weakness 7, Question 4):**\nThe performance degradation of π₀ with force in generalization tasks (Table 2) stems from an out-of-distribution data problem. The base π₀ model was pre-trained on state information that did not include force data. When we concatenate force information to the state input for fine-tuning, the model performs well on in-distribution tasks. However, during generalization to unseen objects and scenarios, the model must rely more heavily on its pre-trained knowledge. Since the force data is an unfamiliar and out-of-distribution signal for the pre-trained π₀, it disrupts its existing priors, leading to a drop in performance. Our ForceVLA architecture is designed to mitigate this by handling multimodal signals more robustly.\n***\n### 3. On Additional Comparisons and Ablations\n**Regarding Comparison to MoE without Force (Weakness 6, Question 6):**\nTo directly assess the value of the force modality within our MoE architecture, we conducted the suggested ablation study where the force input to the ForceVLA model was masked.\n\nThe results are as follows: The success rates were 20% for Plug Insertion, 0% for USB Insertion, 30% for Pump Bottle, and 26.7% for Wipe Board. The Peel Cucumber task was not reported due to time constraints. Compared to the success rates of ForceVLA with force inputs, these results represent a drop of 60% for Plug Insertion, 25% for USB Insertion, 37% for Pump Bottle, and 0.3% for Wipe Board.\n\nIn this ablation study, the common failure modes for ForceVLA w/o force inputs included:\n- During Plug Insertion, the end-effector could only push a short distance in the insertion direction, and the trajectory's curvature was insufficient to properly insert the plug into the socket; alternatively, the end-effector would only move to the plug's surface and perform an arcing push motion near the contact point without making actual contact, similar to overfitting the insertion trajectory.\n- In the USB Insertion task, the gripper's closing width was too large, making it unable to grasp the USB drive, possibly due to overfitting the gripper width and preventing a firm grasp.\n- When pumping the bottle, it repeatedly applied excessive force, causing the gripper to bend, or the off-center press caused the bottle to spring away.\n- When wiping the board, the number of attempts to erase the writing was reduced, leading to most cases where the board was not wiped clean.\n\nWe summarize the likely cause of these failure modes as follows: Without force input, using the FVLMoE module increases the model's total number of parameters, making it more prone to overfitting. During inference, it exhibits many stereotypical behaviors that are misguided by vision, which represents an overfit to the action distribution of the training data. Conversely, the fact that the full ForceVLA model with the additional force modality does not suffer from this overfitting strongly indicates that our architectural design is effective at leveraging force signals to regularize the policy and achieve robust, reactive behaviors instead of merely memorizing training trajectories. We will add this crucial ablation study and analysis to the paper.\n\n**Regarding Related Work on VTLA and FuSe (Weakness 1):**\nWe sincerely thank you for pointing out these highly relevant works. We have studied both papers and agree they are essential for situating our work. We will add a detailed comparison to our Related Work section.\n- Comparison with VTLA: VTLA focuses on sim-to-real transfer using a new simulation dataset and fingertip tactile sensors. Our work is fundamentally different as it is a real-world-first approach, using a new real-world dataset with 6D force-torque data, and our novelty lies in the MoE architecture for online force modulation, not sim-to-real.\n- Comparison with FuSe: FuSe introduces a finetuning recipe to adapt existing generalist models to new modalities. Our contribution is a complete, end-to-end policy architecture. Our focus is on investigating a new architectural paradigm (the MoE) for deep integration.\n\nWe hope these clarifications, analyses, and new experimental results have addressed your concerns. Incorporating these changes will significantly improve the quality and clarity of our paper. Thank you again for your valuable guidance."}, {"review_id": "47CfIia3NO", "reviewer": "Reviewer_hqeq", "summary": "This paper proposes ForceVLA, a novel end-to-end manipulation framework that treats external force sensing as a modality within VLA systems. ForceVLA introduces FVLMoE, a force-aware Mixture-of-Experts fusion module that dynamically integrates pretrained visual-language embeddings with real-time. The paper also introduce ForceVLA-Data, a new dataset comprising synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. ForceVLA improves average task success by 23.2% over strong π0-based baselines, achieving up to 80% success in tasks such as plug insertion.", "questions": "1. Please provide a more thorough discussion contrasting these works with the proposed dataset.\n\n2. Limited discussion on failures: While the proposed method demonstrates favorable performance against existing state-of-the-art, it is unclear why the proposed method fails \"Pump Bottle,\" \"Wipe Board-1 and 2.\" The description of these failures is limited. Could different fusion strategies work better for these categories?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. This paper explores a timely and important direction—integrating force signals into vision-language-action (VLA) models for contact-rich manipulation tasks. The proposed method, a force-aware Mixture-of-Experts (MoE) fusion module, enables dynamic processing and deep integration of force, visual, and language modalities during action generation. This represents a meaningful advancement in the development of multimodal large action models, particularly for tasks requiring nuanced physical interactions. I believe the work would be of interest to the community who is working on multimodal learning, multimodal large action model, robot learning, and contact-rich manipulation.\n2. The authors benchmark their model against state-of-the-art approaches such as pi-0 and demonstrate superior performance in tasks like “Insert USB,” “Insert Plug,” and “Peel Cucumber.” Furthermore, under five challenging settings, ForceVLA consistently outperforms π0, highlighting its robustness and effectiveness across diverse contact-rich scenarios.\n3. The authors conduct ablation studies on various architectural designs of ForceVLA, with a particular focus on the integration of force feedback. This analysis is valuable for researchers interested in incorporating temporal signals into vision-language models.\n\nWeaknesses:\n\n1. One of the key contributions of this manuscript is the introduction of a comprehensive data collection pipeline tailored for contact-rich manipulation tasks. This includes the development of teleoperation tools, data converters, and the release of a new dataset. Such an infrastructure is highly valuable for advancing learning in physically grounded manipulation tasks where force feedback plays a crucial role.\n\nHowever, the paper does not sufficiently discuss the limitations of existing contact-rich manipulation datasets or clearly position its contributions in relation to prior work. Several recent efforts have made significant strides in this space:\n\n- **Fan et al.**, *RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot* (RSS 2023 Workshop), introduced a broad dataset for generalizable robotic skill learning.\n- **Narang et al.**, *Factory: Fast Contact for Robotic Assembly* (RSS 2022), focused on high-speed, precise contact interactions in assembly tasks.\n- **Tang et al.**, *IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality* (RSS 2023), tackled the sim-to-real gap for contact-rich assembly by leveraging simulation and domain adaptation.\n\nA more thorough discussion contrasting these works with the proposed dataset—in terms of task diversity, force sensing resolution, data volume, teleoperation fidelity, and real-world applicability—would strengthen the contribution and help the audience better appreciate the novelty and utility of this new resource.\n\n2. Limited discussion on failures: While the proposed method demonstrates favorable performance against existing state-of-the-art, it is unclear why the proposed method fails \"Pump Bottle,\" \"Wipe Board-1 and 2.\" The description of these failures is limited. Could different fusion strategies work better for these categories?", "was_revised": true, "final_justification": "I appreciate the authors’ efforts in addressing the comments raised in my original review. The rebuttal addressed them. Please ensure to incorporate these comments in the final manuscript.\n\nOverall, the work is important to the community. Specifically,\n- This paper explores a timely and important direction—integrating force signals into vision-language-action (VLA) models for contact-rich manipulation tasks.\n- The authors benchmark their model against state-of-the-art approaches such as pi-0 and demonstrate superior performance in tasks like “Insert USB,” “Insert Plug,” and “Peel Cucumber.”\n- The authors conduct ablation studies on various architectural designs of ForceVLA, with a particular focus on the integration of force feedback. This analysis is valuable for researchers interested in incorporating temporal signals into vision-language models.\n\nThus, I recommend accepting the paper.", "rebuttal": "We sincerely thank Reviewer hqeq for their positive assessment and very insightful feedback. We are encouraged that the reviewer found our work to be a \"timely and important direction\" and a \"meaningful advancement.\" Your constructive comments have been instrumental in helping us clarify and strengthen our contributions. We address your points below.\n ***\n### 1. Regarding the Discussion and Positioning of the Dataset (Weakness 1, Question 1)\n\nWe thank the reviewer for this detailed and extremely helpful suggestion. We agree completely that a more thorough comparison with existing datasets will better situate our work and help the community appreciate its unique utility. As suggested, we have performed a detailed analysis and will incorporate the following discussion into the final version of our paper.\n- Comparison with RH20T (Fan et al.): While RH20T provides a valuable, large-scale dataset, its primary offering is raw, unformatted data with basic reading tools. Our work significantly advances usability and integration by providing not only the raw data but also a pre-processed dataset in the SOTA Lerobot format and the conversion tools themselves. Datasets in the Lerobot format are \"plug-and-play,\" meaning they can be directly loaded to train most models within the powerful Lerobot library. This out-of-the-box compatibility removes significant data-wrangling overhead, greatly accelerating research in this area.\n- Comparison with Factory (Narang et al.): Factory is a powerful toolkit focused on simulating contact-rich interactions at scale, providing virtual assets and controllers to train policies in a simulated environment. Our approach is fundamentally different, as our data is collected entirely in the real world. By doing so, our pipeline and dataset bypass the well-known sim-to-real gap. Our work thus offers a complementary, reality-first methodology that is more directly applicable to physical robot deployment.\n- Comparison with IndustReal (Tang et al.): The approach of IndustReal is notable as it explicitly avoids F/T sensors, instead relying on other cues to manage contact. Our work is built on a contrasting philosophy. We posit that explicit, high-resolution force feedback is a critical signal for dexterous manipulation. Therefore, our ForceVLA-Data prioritizes the deep integration of 6D F/T sensor data. This positions our dataset as an essential resource for developing and benchmarking a distinct class of policies that explicitly leverage force.\n\nIn summary, while these pioneering datasets have pushed boundaries in task diversity (RH20T), simulation (Factory), and sim-to-real (IndustReal), our ForceVLA-Data offers unique contributions in three key areas: out-of-the-box usability with the Lerobot framework, a reality-first methodology, and a focus on explicit 6D force sensing.\n\nWe will add this detailed discussion to a new subsection in our Related Work section. We thank the reviewer again for prompting this valuable clarification.\n \n### 2. Regarding the Discussion of Failure Modes(Weakness 2, Question 2)\n\nWe thank the reviewer for pointing out the need for a more detailed failure analysis. Below, we describe the typical failure modes for our ForceVLA model on the mentioned tasks and address the question about alternative fusion strategies.\n- For \"Pump Bottle\": The most common failure mode for ForceVLA was making contact with the pump head off-center. It is important to note that for the standard task with a bottle seen during training, our model achieved a 100% success rate. The observed failures occurred primarily when we introduced challenging visual variations (occlusions and background changes) during testing to probe the model's robustness.\n- For \"Wipe Board\": The failures for ForceVLA fell into two categories:\n1. Grasping Failure: In some cases, the model failed to pick up the eraser. We attribute this to visual perception challenges, as the eraser was placed far from the base camera, resulting in a low-resolution image, and its color was similar to the black table, making precise localization difficult.\n2. Timeout due to Meticulousness: Interestingly, some failures were due to timeouts. The model would successfully erase ~90% of the writing but would then spend a prolonged period making repeated, fine-grained attempts to erase a tiny remaining smudge. While this is technically a failure by our timeout metric (5 minutes), this behavior suggests the policy is highly sensitive to the task goal, perhaps overly so.\n- On Different Fusion Strategies: This is an excellent question. For the failures we observed (e.g., grasping the low-resolution eraser, off-center pressing under visual distraction), we believe the root cause lies more in the limitations of the visual perception backbone rather than the FVLMoE fusion module itself. Our MoE is designed to dynamically weigh inputs, but it cannot overcome fundamental ambiguity or a lack of fine-grained detail in the upstream visual stream. We hypothesize that an alternative fusion strategy would likely face similar challenges if the input perception is noisy. However, we agree that exploring fusion mechanisms that can explicitly model visual uncertainty is a very promising direction for future work. We will add this discussion to the limitations section of our paper.\n***\nOnce again, we sincerely thank you for your thoughtful and constructive review. Your feedback has been invaluable in helping us strengthen the paper, and we hope our responses and planned revisions have fully addressed your concerns. Thank you for your time and guidance."}, {"review_id": "Da4fla9VRp", "reviewer": "Reviewer_4bXA", "summary": "This paper proposes ForceVLA, a Vision-Language-Action model enhanced with 6D force sensing. The core module, FVLMoE, is a force-aware Mixture-of-Experts that fuses force, vision, and language modalities during action generation. A new dataset, ForceVLA-Data, is collected for contact-rich tasks. Experiments on 5 real-world tasks show improved performance over π0 baselines.", "questions": "-Why are there no simulation experiments? Could that help with reproducibility or scalability?\n-- π0-fast w/f performs best in the Unstable Socket task. Does this mean that simple fusion is sufficient if the pre-training data contains enough torque sensor data? Can the authors share insights on π0-fast competitiveness with sufficient torque sensor training data?\n-Can ForceVLA generalize to low-cost robots without precise F/T sensors?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\nNovel use of force sensing as a first-class modality in VLA models.\nThe FVLMoE fusion module is a technically sound idea.\nSolid improvements in task success rates (+23.2%) across five contact-rich tasks.\nWeaknesses\nNo simulation experiments, which limits reproducibility and accessibility.\nIn the \"Unstable Socket\" task, π0-fast w/f performs best, suggesting that a simple connection can be effective if the pre-training data contains enough torque sensor perception information.", "was_revised": true, "final_justification": "The rebuttal addressed my main concerns, and with no major unresolved issues, my overall assessment and recommended score remain unchanged.", "rebuttal": "We thank Reviewer 4bXA for the constructive feedback and for raising important questions about the interpretation of our results and the broader applicability of our work. We address each point below.\n ***\n### 1. On Simulation Experiments, Reproducibility, and Scalability(Weakness 1, Question 1)\n\nWe appreciate the reviewer's point regarding simulation experiments. Our primary focus for this work was to tackle the challenges of force-aware manipulation directly in the real world, as physical interaction and multi-body contact dynamics are central to our problem. We believe that using real-world data ensures our findings are directly applicable to physical deployment scenarios.\nWhile we agree that simulation offers significant benefits for scalability and reproducibility, developing a high-fidelity simulation for these diverse, contact-rich tasks was beyond the scope of our current resources for this submission. However, we concur that large-scale training with generated data in simulation is a very promising direction for expanding the range of tasks our model can address. We see this as a key area for future work and will add this discussion to the paper.\n \n### 2. On the Performance of π₀-fast in the \"Unstable Socket\" Task(Weakness 2, Question 2)\n\nThis is a very keen observation, and we thank the reviewer for asking for a deeper insight into this interesting result.\nOur analysis reveals a nuanced reason for this behavior. We have consistently observed that π₀-fast models, due to their architecture, often exhibit stuttering, pausing mid-air, or significant delays between actions. Paradoxically, this frequent pausing becomes an advantage for the specific \"Unstable Socket\" task. When the end-effector pauses while in contact with the plug, the likelihood of the unstable socket sliding or wobbling decreases significantly. This effectively reduces the difficulty of this particular generalization task to that of a stable socket, creating the impression of superior performance for the π₀-fast model.\nHowever, for general contact-rich, fine-manipulation tasks, we argue that π₀-fast is not as competitive. Its autoregressive decoding nature limits its inference speed, which in turn hinders its ability to react promptly to real-time force changes and adjust its trajectory flexibly. We will add this important nuance to our analysis section to prevent misinterpretation of this result.\n \n### 3. On Generalization to Low-Cost Robots without Precise F/T Sensors(Question 3)\n\nThank you for this forward-looking question about practical deployment. Due to time and hardware constraints, our current experiments are limited to a single robot platform. However, we strongly agree that demonstrating generalization to more common, lower-cost platforms is crucial. To promote wider practical deployment and help democratize force-aware manipulation research, we are actively assessing ForceVLA’s adaptability and performance on lower-cost platforms equipped with external or retrofitted force sensors. \n***\nWe hope our responses have clarified these points and addressed your concerns. Thank you again for your valuable feedback."}, {"review_id": "C03rdw3BLh", "reviewer": "Reviewer_7Vfu", "summary": "The paper presents ForceVLA, a visual-language-action (VLA) model enhanced with force sensing and feedback. Force signals are perceived online and integrated into the model to guide action generation. To enable force awareness, the authors introduce a force signal processing layer and a mixture-of-experts (MoE) architecture for effective force integration. A teleoperation system is developed to collect data with force annotations, resulting in a curated dataset spanning five diverse tasks. Extensive experiments on these contact-rich tasks demonstrate the effectiveness of ForceVLA and its ability to generalize across different scenarios.", "questions": "- Does the teleoperation setup provide force feedback to the human operator? What specific challenges do the authors face when teleoperating these contact-rich tasks? Additionally, what are the teleoperation success rates for each task?\n- Is the final ForceVLA model trained as a single multi-task policy, or are separate task-specific policies used? \n- While the method is framed as a VLA model, the language component appears limited to providing task instructions for a small number of predefined tasks. Since the model is not aimed at open-world generalization, developing the approach as a vision-based, force-aware policy may be more accurate and appropriately modest than purposely integrating \"L\" to make it align with popular VLAs. \n- How is the base policy $\\pi_0$ used in the training pipeline? Is ForceVLA trained from scratch, or is it fine-tuned from $\\pi_0$?\n- The model is trained on only a few hundred trajectories. Are these data sufficient to avoid overfitting, especially given the complexity of the tasks? What specific mechanisms or design choices encourage generalization, particularly given the relatively small training dataset?\n- Have the authors considered comparing Diffusion Policy or 3D Diffusion Policy? While these are not VLAs, they represent competitive approaches for visuomotor control.", "limitations": "- The overall quality of the results is somewhat restricted. Some task executions exhibit suboptimal behaviors such as unsmooth trajectories, retries, and dropped objects, which raise concerns about robustness and precision.\n- The originality of the method is a little bit restricted. Both feature fusion and MOE have already been proposed in previous works.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n- Strong motivation. The paper is well-motivated. Contact-rich tasks pose significant challenges for standard VLA policies, making them ideal candidates for evaluating the integration of tactile and force feedback.\n- Well-designed method. The proposed approach is thoughtfully designed. The force signal processing and fusion mechanisms are clearly described in the method section. The fusion strategy, along with the MoE architecture, is a reasonable and effective choice for handling heterogeneous information.\n- Effective and generalizable. ForceVLA demonstrates strong performance on challenging contact-rich tasks such as plug insertion, USB insertion, and cucumber peeling. The authors further validate the model’s generalization capability across variations in object type, insertion height, and external disturbances through a comprehensive set of experiments.\n\nWeaknesses\n\n- Questionable task difficulty and quality. Some task setups raise concerns about difficulty and realism. For example, in the plug insertion task, the plug is initially half-inserted into the socket, significantly reducing the complexity—requiring the robot to simply reach and make a small movement to complete the task. In the demonstration video, the robot’s end-effector trajectory appears quite unsmooth, and it relies on trial-and-error to locate the correct contact point. The USB insertion task shows more impressive results; however, the trajectory near the socket is still jerky, and the robot even drops the USB at one point and needs to re-grasp it. In contrast, tasks like cucumber peeling and board wiping show better performance, though they do not require high precision in position or force control.\n- Task design. As noted, the initial configuration of the plug being half-inserted undermines the challenge of the task. In the board wiping task, the robot can succeed simply by applying a large force, without needing fine control over the force applied. A more demanding task—such as wiping a bristle bottle—would better justify the need for precise force sensing and execution. Additionally, many of the generalization experiments focus on variations in visual appearance or visual disturbances, which are typical in standard VLA setups and do not highlight the unique benefits of force modeling. \n- The website could be more organized.", "was_revised": true, "final_justification": "ForceVLA presents a framework for collecting vision-tactile data and a VLA architecture for policy training. Leveraging vision and tactile signals, the authors have demonstrated impressive capabilities in completing fine-grained tasks with heavy occlusions, where tactile information would play a crucial role.  In summary, the paper explores a promising direction to build stronger robot policy, where the specific focus is integrating force information, and has demonstrated its superiority via thorough experiments. It contributes valuably to the community and deserves acceptance.", "rebuttal": "We sincerely thank Reviewer 7Vfu for their extremely thorough and detailed review. We appreciate the positive comments on our motivation and method design. Your critical feedback on task design, evaluation, and the framing of our contributions is invaluable, and it has pushed us to significantly clarify and strengthen our paper. We address each of your concerns below.\n ***\n### 1. On Task Design, Difficulty, and Generalization\n\nThis section addresses the reviewer's primary concerns about the quality and design of our tasks and generalization experiments.\n\n**Regarding Task Difficulty and Realism (Weakness 1 & 2):** We appreciate the reviewer's concerns about task design and would like to provide more context.\n- On the \"Plug Insertion\" setup: We understand the concern that starting with the plug half-inserted reduces complexity. We initially attempted a full \"pick-move-insert\" sequence. However, due to hardware limitations, our gripper's frictional force was insufficient to hold the plug firmly against the large insertion forces required. We therefore opted for the \"push-to-insert\" task design. This setup introduces its own unique challenges that are highly relevant to force control: without a firm grasp, the plug is prone to wobbling and shifting, requiring the policy to be highly sensitive to instantaneous force changes to maintain a stable push. We will add a discussion of this hardware limitation and the resulting design choice to the paper. When hardware permits (e.g., with a stronger gripper), we plan to explore the full sequence in future work.\n- On \"Unsmooth Trajectories\": We apologize for this impression. The videos on our website were sped up 2-3x for brevity. The original trajectories are smoother, and we will clarify this on the website. Regarding \"trial-and-error,\" we acknowledge this reflects the current capabilities of the model and that achieving smoother, more \"ballistic\" motions is a key challenge for future work.\n- On \"Board Wiping\": We agree that if the task could be solved by simply applying a large force, its value would be diminished. However, this is not feasible in our setup. Applying excessive force pushes the whiteboard out of its fixed position and beyond the robot's workspace, leading to task failure. Therefore, controlled force application is necessary to wipe the board effectively while keeping it in place.\n- On a \"More Demanding Task\": The suggestion of wiping a bristle bottle is excellent. This is a great idea for a more challenging benchmark, and we will consider it for future work.\n\n**Regarding the Benefits of Force Modeling in Generalization (Weakness 2):** Thank you for pointing out the need to better highlight how our generalization experiments test force modeling. While some experiments involve visual changes, many are designed to create novel physical dynamics:\n- For Pump Bottle, generalization to bottles of different heights and pumps with different travel distances requires the policy to use force feedback. For shorter bottles or longer pumps, the robot must press lower, using the upward reaction force to determine when the press is complete or if it has exceeded a safe limit.\n- For Plug Insertion, using plugs of different sizes and shapes introduces complex new force dynamics. Smaller plugs demand more precise directional control during contact, while taller plugs require fine control over the timing and magnitude of the force. The \"unstable socket\" generalization specifically introduces random, unpredictable dynamic forces (sliding, wobbling), forcing the model to rely on precise force sensing and immediate adjustments to counteract the instability. The visual occlusion task for this setup similarly forces a greater reliance on force modeling over vision.\n\nWe will revise our experimental section to make these force-specific challenges much clearer.\n ***\n### 2. On Methodological & Training Details\nThis section addresses the reviewer's specific questions about our implementation.\n\n**Regarding Teleoperation Details (Question 1):** Although the human operator does not receive direct haptic feedback, they are provided with real-time visual feedback of the forces within their Quest3 VR headset. The 6D wrench is visualized as a dynamic force vector at the TCP, allowing the operator to see and react to contact forces. The main challenge during teleoperation (e.g., for USB insertion) is the low resolution of the headset's display, which can make fine details hard to see from a distance. Operators sometimes mitigate this by looking at the real setup with their own eyes. We have not formally tracked the teleoperation success rate, but anecdotally, we perform about 60 teleoperation trials to collect 50 successful demonstrations, suggesting a success rate of approximately 85%.\n\n**Regarding Single vs. Multi-task Training (Question 2):** Thank you for this question. In our main experiments (Figure 5), we trained separate, task-specific policies. However, we also conducted multi-task joint training, and these results are reported in Section D (Table 5) of our supplementary material. The results show that our model performs well both as a single multi-task policy and as separate task-specific policies. We will add a pointer to these results in the main paper.\n\n**Regarding Base Policy (π₀) Usage (Question 4):** We initialize our model with the pre-trained weights of π₀. During training, we use the LoRA method to fine-tune the original π₀ parameters (such as the Gemma model and action expert), while we train the new parameters of our proposed FVLMoE module from scratch. We will clarify this training procedure in the methodology section.\nRegarding Dataset Size and Generalization (Question 5): We agree that a few hundred trajectories is a relatively small dataset. We believe our strong generalization results demonstrate that the model is not merely overfitting. We encourage this generalization through two main mechanisms:\n1. Implicit Regularization from Pre-training: Fine-tuning a powerful VLA like π₀ provides a strong prior that helps prevent overfitting.\n2. Data Diversity: During data collection, we consciously introduce variations in object starting positions, interaction angles, and approach directions to enrich the training data. We also randomize initial object positions during testing.\n ***\n### 3. On Positioning, Originality, and Broader Comparisons\nThis section addresses the higher-level concerns about our work's framing and its relation to other methods.\n\n**Regarding the VLA Framing and Language Component (Question 3 & Limitations):** This is a fair point. While the language instructions in our main experiments are for a predefined set of tasks, our goal is to build towards general-purpose robots that can handle diverse tasks at a macro level while also executing fine-grained, contact-rich motions at a micro level. Our multi-task training results (in Appendix D), which require language understanding to differentiate tasks, show that our model has potential for open-world generalization. We chose the VLA framework to align with this long-term vision, rather than developing a more narrowly-focused, vision-force policy. While we acknowledge we have not demonstrated large-scale language generalization due to time constraints, we believe our work is a critical step in that direction.\n\n**Regarding Originality (Limitations):** While the core components we use (feature fusion, MoE) have been explored in other contexts, we believe our primary novelty lies in the specific architectural design and first-of-its-kind demonstration of integrating these techniques for online, 6D force-based VLAs operating in complex, real-world contact tasks. This specific problem domain and our successful application remain underexplored.\n\n**Regarding Comparison to Diffusion Policy (Question 6):** Thank you for the suggestion. We agree that Diffusion Policy and 3D Diffusion Policy are strong baselines for visuomotor control. For this initial work, we prioritized a direct comparison with the π₀ VLA family, as our method is a direct extension of the VLA paradigm. Given the time constraints of the rebuttal period, we were unable to implement and evaluate these new baselines, but we recognize their importance and will include them in future iterations of this work.\n\nWe also thank the reviewer for the feedback on our project website. We will update it with a more organized and polished design to better present our methodology and results.\n***\nWe hope these detailed responses have addressed all your concerns. We are grateful for the thorough review, which will undoubtedly help us produce a much stronger final paper."}]

Author rebuttal:

Meta-review: The paper proposes ForceVLA, a force-aware vision-language-action model for contact-rich manipulation using a Mixture-of-Experts module routed by estimated force levels. The approach supports both vision-only and vision+force settings, and is validated on peg-in-hole, insertion, and rotation tasks.

Reviewers are generally positive (one Accept, three Borderline Accepts), highlighting the motivation, real-robot validation, and modular design. Concerns on novelty and experimental depth are addressed in the rebuttal and final remarks with added ablations, failure analyses, and task justifications.

Overall, this is a timely and practical contribution to multimodal policy learning. The AC recommends acceptance, with the expectation that the camera-ready version further strengthens experimental analysis, related work, and presentation clarity.
Decision: Accept (poster)