CogVLA: Cognition-Aligned Vision-Language-Action Models via Instruction-Driven Routing & Sparsification
Authors: Wei Li
Renshan Zhang
Rui Shao
Jie He
Liqiang Nie
Venue: NeurIPS 2025 poster ()
Keywords: Vision Language Action models; Instruction-Driven; Vision-Language Models; Vision Sparsification
Primary area: applications
TL;DR: We propose CogVLA, a biologically inspired cognition-aligned VLA framework that leverages instruction-driven routing and cross-modal sparsification to achieve state-of-the-art performance with significantly reduced computation and latency.

Abstract: Recent Vision-Language-Action (VLA) models built on pre-trained Vision-Language Models (VLMs) require extensive post-training, resulting in high computational overhead that limits scalability and deployment. Existing sparsification strategies—such as Mixture-of-Depths, layer skipping, and early exit—fall short by neglecting the semantic coupling across vision-language-action modalities, and focusing narrowly on intra-LLM computation while overlooking end-to-end coherence from perception to control. To address these challenges, we propose **CogVLA**, a Cognition-Aligned Vision-Language-Action framework that leverages instruction-driven routing and sparsification to improve both efficiency and performance. CogVLA draws inspiration from human multimodal coordination and introduces a 3-stage progressive architecture. 1) **Encoder-FiLM based Aggregation Routing (EFA-Routing)** injects instruction information into the vision encoder to selectively aggregate and compress dual-stream visual tokens, forming a instruction-aware latent representation. 2) Building upon this compact visual encoding, **LLM-FiLM based Pruning Routing (LFP-Routing)** introduces action intent into the language model by pruning instruction-irrelevant visually grounded tokens, thereby achieving token-level sparsity. 3) To ensure that compressed perception inputs can still support accurate and coherent action generation, we introduce **V‑L‑A Coupled Attention (CAtten)**, which combines causal vision-language attention with bidirectional action parallel decoding.
Extensive experiments on the LIBERO benchmark and real-world robotic tasks demonstrate that CogVLA achieves state-of-the-art performance with success rates of 97.4\% and 70.0\%, respectively, while reducing training costs by 2.5$\times$ and decreasing inference latency by 2.8$\times$ compared to OpenVLA.

CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification
Wei Li Renshan Zhang Rui Shao' Jie He Liqiang Nie
School of Computer Science and Technology, Harbin Institute of Technology. Shenzhen 1iwe120240stu. hit, edu. shaoruihhit. edu. cn attps://jiutian www.aannvv.iiiiii... github. crittub..o/Copp io/CogVLA -page
Instrutiion Place the toy bear drawer, then put the orange culoe on the plate,
Performa ccherent
(4) Initial Obearvation
Torged instruction
obsects infer action
SONAA
PEHAA
352
[1) Paiting Vicies Carprresian Nethoals VIA
Cayst
Takia staps
Ramoniaar (50
of CogyLe and OpeniiLA
and training Eificiency (butbars)
Katee en LIMEROO benchieaz
Figure 1: Overview of proposed CogVLA. Traditional VLA models process initial observations (Fig. without vision compression, leading to high computational cost. As shown Fig. and (d), existing compression methods retain irrelevant inputs and fail to focus instruction- relevant targets c VLA employs EFA -Futing and LFF-Routing to sparsify visual inputs based instruction relevance. Comparing Fig-(e) and (e). CAtten further enhances logical consistency and action coherence for final targeted objects (f), Fig. and (ig.(h) illustrate the architectural innovations of CogVLA and its superiority in efficiency and performance.
Abstract
Recent Laioon-LanguageAAction (VLA) models built on pre-trained Vision- Language Models (VLMs) require extensive post-training, resulting in high com- putational overhead that limits scalability and deployment. Existing sparsification strategies-such Mixture-of- layer skipping, and early exit-fall short by neglecting the semantic coupling across vision-language-action modalities,
"Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

and focusing narrowly on intra-LLM computation while overlooking end-to-end coherence from perception control. To address these challenges, we propose CogVLA, Cognition-Alignet Vision-Language-Acttinn ages instruction-driven routing and sparsification to improve both efficiency and performance. CogVL.A draws inspiration from human multimodal coordination and introduces a 3 3-stage progressive architecture. 1) Encoder-FiL.M based Ag- gregation Routing (CFA-Routing) injects instruction information into the vision encoder to selectively aggregate and compress dual-stream visual tokens, forming instruction-awee aware latent representation. 2) Building upon this compact visual encoding, LLM-FiLM based Pruning Routing (LFP-Routing) introduces action intent into the language model by pruning instruction -irreeevanneeenntt visually grounded tokens, thereby achieving token-level sparsity. 3) To ensure that compressed percep- tion inputs can still support accurate and coherent action generation, we introduce V-L-A Coupled Attention (CAtten), which combines causal vision-language attention with bidirectional action parallel decoding. Extensive experiments the LIBERO benchmark and real-world robotic tasks demonstrate that Cog VLA achieves state-of-the-arr performance with success rates of 97.4% and 70.0%, re- spectively, while reducing training costs by 2.5x and decreasing inference latency by 2.8 compared to OpenVLA.
framework that lever-
Introduction
84 85 research has advanced rapidly, fueled by the rich visual and linguistic representations provided by powerful pre-trained Vision-Language Models (VLMs) I 42 22 2 (88. Leveraging these foundational models, the VLA paradigm
Vision-Language Action (VLA) 88
36
30
progressing toward end-to-end robotic control and embodied intelligence, enabling agents to comprehend natural language instructions, perceive complex scenes, and perform manipulation tasks with minimal task-specific engineering. Pioneering works such as RT-2 [ Octo (65) OpenVLA 30)
To [6. and Wo.5 (27 have demonstrated the potential of this paradigm.
However, aligning the high-dimensional multimodal features output by VLMs with continuous action spaces remains computationally expensive 88 7T TO 5 30). Standard fine-tuning and joint training procedures often entail substantial memory consumption, high FLOPs overhead, and extended training times, severely limiting scalability and practical deployment on resource- constrained platforms. For instance, fine-tuning, 7B VLA model 29 with action chunking single-task dataset from the LIBERO benchmark [4] consumes over 600 GPU hours (using 80G A A100 GPUs), incurring significant computational costs. Although techniques such as Mixture-of- Depths [51 77 44, 13,, layer skipping [82 72), and early exit [74 17 have been proposed to sparsify and accelerate the model training and inference, these methods primarily focus on computation optimization within language models, overlooking the semantic coupling across perception, language alignment, and action decoding. This modular optimization paradigm often leads cross-modal semantic degradation, manifesting as follows: i) visual compression within encoders discards task-relevant fine- mne-griined features, ii) token skipping within LLMs disrupts the contextual coherence necessary for reference resolution, and iii) action generation lacks causal reasoning over multimodal state transitions.
humans exhibit a highly optimized and efficient multi- modal coordination mechanism during manipulation. For example, when receiving the instruction "place the red cup at the corner of the table," " the human Visual and Attention System (VAS) selectively focuses (7) on the color attributes of the cup and the spatial structure of the table. Concurrently, the Supplementary Motor Area (SMA) injects task-relevant action intentions (61] derived from key semantic associations (e.g., rred-cup-comer"" into the visual processing stream, while the Premotor Cortex (PMC) dynamically integrates both visual and linguistic information to plan coherent motion trajectories. This organic unification of perception, reasoning, and control results in remarkable task efficiency. Inspired by this, we propose CogVLA, Cognition-Alignee framework based on Instruction-Drivee ing modular pipelines, CogVIA establishes tet---maaniiiconssseeeeee joint optimization mechanism across vision, language, and action modalities, reinforcing cross- cross-modal coherence while improving computational efficiency.
From a cognitive science perspective 52 32
LaaiionLangugg--ee Action Routing & Sparsification, as shown Fig. (f. Unlike exist-

---

Specifically, CogVLA adopts a 3-stage progressive design to jointly enhance computational efficiency and task performance, as shown in Fig. (d) and (e): 1) Encoder-FiLM based Aggregation Routing (EFA-Routing): To alleviate visual information redundancy and achieve VAS-like visual focus, EFA-Routing compresses visual tokens to 25% of the original input scale, guided by task-specific instructions. This process begins by dynamically encoding the instruction into modulation parameters that guide the aggregation of visual tokens within the visual encoder. Subsequently, the outputs from different encoder branches are adaptively fused to produce cross-branch representations that are semantically aligned with the given task. 2) LLM-FILM based Pruning Routing (LFP-Routing): Building upon the aggregated visual encoding, LFP-Routing learns novel, instruction- aware sparsity pattern to prune visual tokens within the language model. By emulating the functionality of SMA, which injects action intentions into visual features, the mechanism selectively skips attention com- putations over 50% of task-irrelevant tokens. As a result, it significantly reduces the computational burden of the language model and effectively minimizes latency in action generation. 3) V-I-- Coupled A Attention (CAtten): To ensure that the compressed visual inputs retain the capacity to support accurate and coherent action sequence, CAtten introduces coupled attention mechanism inspired by PMC: i) Cross-modal causal attention is applied between the V-L-A interaction layer to preserve temporal reasoning capabilities; ii) Unidirectional attention is employed within the V-L layer to ensure semantic consistency, where visual features have been pre-enhanced with task-specific language intent; iii) Bidirectional attention utilized within the Action layer to enhance the coherence of action sequences and enable efficient parallel decoding.
We conduct comprehensive evaluations of CogVLA the LIBERO benchmark and real-world robotic manipulation tasks. Experimental results show that CogV achieves state-of-the-ar task success rates while reducing end-to-end computational costs significantly, as shown in Fig. D (g)
and (h). Ablation studies further validate the complementarity and synergistic effect of the routing modules and the coupled attention mechanism. Our main contributions are summarized as follows:
We propose Cog, Cognition-Alignee Vision- Sasion-Lnguugg--ae Action framework inspired by human multimodal coordination, which establishes a biomimetic 3-stage architecture: "VAS (visual information focusing) SMA (semantic intent filtering) PMC (action sequence planning)."
We develop synergistic EFA-Routing and LFP-Routing, enabling instruction-driven vision sparsi-
fication in perception-reasoningg pipelines.
We formulate CAtten ensuring cross-modal logical consistency and temporal action coherence in doubly compressed multimodal representations.
Through extensive experiments on the LIBERO benchmark and real-world robotic tasks, we demonstrate the superior performance and efficiency of CogVLA,
2 Methods
2.1 Preliminary: Parallel Decoding in Action Chunk
We consider a sequence prediction setting where a Vision-Language-Action (VLA) model outputs a sequence of actions across K future timesteps. Traditional autoregressive (AR) decoding predicts actions sequentially, whereas parallel decoding enables simultancous prediction of all actions within the chunk, improving inference efficiency and supporting scalable deployment.
Action Chunk, Given the current input context X = (I.t) t) E RM+T, which comprises the visual observation and task instruction, the model predicts chunk of K future actions:
|ao: , ax-1] E R R
(1)
Here, D denotes the dimensionality of each atomic action (e.g., D = 7 for 3-DoF translation AT, 3-DoF rotation AR, and binary gripper control).
Autoregressive Decoding. In causal autoregressive decoding, the action sequence is generated incrementally. For each timestep i (k)
O,..., K 1, the atomic action vector a; RD is produced token-by baken-by-token, with each token a, conditioned on the preceding tokens and previously actions:
a; la, a ,.... ,...,a ( a.
(1) (2)
a fAR( (X, (ar)rei, ((2:k-1)]
(2)
This decoding necessitates K x D forward passes, introducing latency from token-level dependencies.
3

---

V-L-A Coupled Attention (Stage 3)
Large Language Model
LFP-Routing (Stage
Pruning Router
Dusl Aggroptiie
Appregation Router [Stage 1-Step 2)
Itpat
(rppu dene votioon
Weighting
Weighting
Encoder-FiLM
WLLP
Encoden-FILM
AR-15 122,
Apprapete (Stegsi-Stap
Aggingate (Stge E.0epp
BGrooprr Cissevattion
"Place Into the drenee" Impalt Instruction
Obsurvation
Agg regeeiitiin
into drawns.
Observation Tolees
Agarepation
SigLip
DINOv2
Figure 2: Overview of CogVLA Framework. CogVLA employs cognition-aligned instruction- driven routing & sparsification strategy for efficient action chunk prediction. Inspired by human multimodal coordination, it integrates task-guided visual aggregation, semantic pruning, and coherent decoding, ensuring efficient cross-modal representation alignment from perception to control.
Parallel Decoding. In contrast, parallel decoding eliminates the sequential dependency. The model receives the input observation embeddings X along with K empty placeholder embeddings:
X X, Og. .,.... 0x- 1] IR. TBM+++KXDD
where 0, E R denotes a learnable zero-action embedding. Under a bidirectional attention scheme (instead of causal masking), the decoder jointly produces all future actions in a single pass:
Sparlllel X)
2.2 CogVLA: Framework
29 primarily focus on lightweight computation within isolated stages of action chunk prediction VLA models, often leading cross-modal semantic degradation due to modular disconnection. To address this, we propose CogVLA, a ingnition-alignee framework that enhances both efficiency and performance via Instruction-Drivee Routing & Sparsification.
Recent methods 74,
As illustrated in Fig. the framework operates through a 3-stage progressive architecture in- spired by human multimodal coordination. In Stage 1, CogVLA incorporates N vision encoders (Encı,.... Enc v } that extract visual tokens from image observations [(), Each encoder is modulated by the instruction tr (obtained via LLM embedding layer) through an Encoder-FiLA module:
Encoder-FiLM, (I())
tr),
==
(5)
agg
where Vags denotes the aggregation token for the i-th encoder. These modality-specinn aggregated tokens are dynamically dual-aggregated via an instruction-conditionee. routing mechanism:
Vagg
O
ags
(6)
[ai.... [xl
Softmax MMLProueett.
In Stage 2, the dual-agggegation tokens are injected into the LLM, where EFP-Ronting selectively filters out iustruction-irrelevaa visual tokens. This instruction-driven sparsification enables token- level efficiency by reducing redundant attention computation, aligning the retained tokens with task-relevant semantics. The filtered representation is then processed by the proposed CAtten module to produce the task-aligned representation in Stage 3. The module operates as follows:
Z
Zi tz))
(8)
where Z,, and t denote the visual and instruction input tokens at the (1 + 1)-th transformer layer, respectively, with Zn = Vagg as the initial state. Finally, action chunks are decoded in parallel using the compressed multimodal context combined with placcholder action tokens:
A fparallel ())= Fparatiel (Zo, to. .x......00
0k-1l)
(9)

---

Stage EFA Routing
(b) Stage LFP- R-outing
Aggregation
FFN
Self-Attention
FILM
Task Task-Guided Guided Pruning Router
Self- Attentitenio
Observation Tokens
Asprogation
Aggll-ggrggatoon
Stage CAtten other methods
Dual Aggregatggggnnn Tokens
tokens
image
Discreta Parallal
Contibuna Rocale Decoding
OpenVILA
OpenVLA-OFT
COT-VLA
Ours (CAtten CogVLA)
SLUM-FIUM
Figure 3: Illustration of 3-Stage Progressive Design. CogVLA emulates human multimodal coordination via instruction-driven routing and sparsification. EFA-R PFAA-Rouing (Stage 1), LFP-Routing (Stage 2), and CAtten (Stage 3) correspond to the VAS, SMA, and PMC, respectively. (C). highlights the advantages of CAtten over prior attention mechanisms in combining mni-&biidirectional attention, injecting action intent, enabling parallel decoding, and leveraging sparse visual tokens.
Through this progressive design across Stage 1-3, CogVLA realizes instruction-driven sparsification and routing across the vision-language-attinn pipeline, effectively reducing computational overhead while preserving task-relevant semantics and enhancing cross-modal reasoning fidelity.
2.3 CogVLA: Cognition-Aligned 3- 3-Stage Progressive Design
As illustrated in Fig. 3. CogVLA adopts a 3-Stage Progressive Design, which emulates the human't optimized coordination during manipulation tasks: EFA EFA-RRuting mimics the VAS by selectively aggregating visual tokens conditioned on task-specific instructions, thereby achieving focused percep- tion. LFP-Routing emulates the SMA by introducing action intentions into the visual context within the language model, selectively pruning irrelevant tokens to achieve instruction-driven token sparsity. CAtten simulates the PMC by dynamically integrating compressed multimodal representations, ensuring cross-modal logical consistency and temporal coherence in action decoding.
2.3.1 Encoder-FiL.M based Aggregation Routing
Step Intra-encoder Aggregation. To aggregate visual information and enable instruction Instruction-guided representation learning, we introduce Mncoder-FiLLM.. which dynamically consolidates observation tokens into aggregation tokens based on task-specific instructions. The language instruction t, modulates visual tokens
( and aggregation tokens (2)
Vagg within each visual encoder branch:
SEA (I
(i) agg trr) (1 (t,))) )) @ (elf-Au
hee (i) + (t(t,)))
(10)
Aggregate(FFN(((AA)
(i)
(
agg
where 7/s and 3; denote the FiL. scale and shift vectors conditioned on t,, and C represents (t)
element-wise multiplication. Throuuh iterative visual encoder blocks, the aggregation token vage adaptively integrates instruction-relevaan information from observation tokens while discarding redundant information. Consequently, only the final Vagg retained while the image tokens I are discarded, effectively reducing the number visual tokens to 25% of the original size.
(6)
Step 2: Cross-encoder Aggregation. To integrate the aggregated visual representations from two heterogeneous vision encoder branches (SigLIP and DINOv2), we design an instructioncconditionn aggregation routing gate that computes fusion weight E (0, 1) based on the input language

---

instruction. Rather than statically assigning equal contributions, the fusion ratio is dynamically predicted for different tasks to reflect instruction-depenae visual preferences:
a = Sig
b1)) + b2)
(11)
where W1, W2 are trainable weight matrices, b1, b2 are biases, and a denotes the GeLU 24 non-linearity. The final dual-aggregated visual token is computed as:
V
SigLIP + - a) DINOv2 age
Vags ags
(12)
This instraction-condiiione aggregation routing allows the model to adaptively balance visual features from different encoders based on the semanties of the instruction, promoting more effective cross-modal fusion, as shown in Fig. and Fig. 3 (a). At the first transformer layer, we denote
Zo Vagg as the dual-aggregation visual tokens and to as the instruction tokens, which serve as the initial inputs for Stage 2.
2.3.2 LLM-FiLM based Pruning Routing
we recognize that EFA-Routing (Stage 1) aggregates features across all image tokens, potentially retaining redundant or semantically irrelevant visual information. To further reduce computational overhead and steer the visual representation toward the intended action semantics, we propose a lightweight LFP-Routing module prior injecting visual context into the large language model, as illustrated in Fib.
Motivated by sparse token routing techniques 77 46 72).
Given the dual-aggregation tokens and the corresponding task instruction transformer layer 1, denoted as Zi and tÜ respectively, LLM-FiLM performs a semantic-aware modulation as follows:
STP [z; t2) Prune (LMM(t))) @ Z,) (t)m
(13)
Z++1 FFF(SSlf-Au((fpp( + Zi
where (-)M(-) and (-LM(-) denote instruction--condiionne scaling and shifting functions, respectively, both implemented as lightweight MLPs. The Prune(-) operation selectively discards tokens with low
task relevance, producing filtered representation that maintains critical visual semantics.
We introduce Task Bask-Guided Pruning Router to implement the Prune(-) operation. This module filters tokens based on their instruction-awen relevance, preserving only those most critical to the task. At Transformer layer I, routing weights R2 computed for each visual token Z using an MLP:
R = MIP(Z/))
(14)
We define token retention ratio B, and determine a relevance threshold Pa as the B-th percentile of the routing weights at layer Only tokens whose scores exceed P are preserved. Formally:
Z/41 [.] fsr [Z. ti) + Z/ if R P!d otherwise
(15)
Z
where fsr(-) represents the self-attention and feed-forward operations within the current layer. The hyperparameter 3 [0, governs the sparsity level by controlling the proportion of retained tokens.
2.3.3 V-L-A Coupled Attention
To maintain semantic consistency and temporal coherence under compressed multimodal inputs, introduce V-L-A Coupled Attention (CAtten), a biologically inspired mechanism grounded in the functional role of the PMC for planning and coordination. As shown in Fig. 3 (c), CAtten hierarchically combines causal and bidirectional attention across vision, language, and action streams. At the (-th transformer layer of the LLM, the input multimodal token sequence is defined as:
[z, ti, A1] M M+T+KXD
(16)
where A; ja,.. 1.......a. a
denotes the action chunk. CAtten operates in three consecutive stages:
Causal Vision-Language Attention. To preserve instrrctionnccondiiioeed visual reasoning, causal attention is applied over the concatenated vision-language token segment:
Attnvr. (Zz, t/]) Softmax
ZZ
[Z
[Z.+
V
Vilssll [Z, tz]
(17)
Vd
where MV VLL
R (MM+T) is lower-trianuular mask within vision-language tokens.
cansal

---

Table Simulation Experimental Results. Comparison of task success rates (SR) and their ranks
(RK) on the LIBERO benchmark across four task types. "q" indicates our reproduced results.
Method
Spatial
Object
Goal
Long
Average
SR f
RK +
SR
RK +
SR
RK
SR
t
RK
SR t
RK +
Diffusion Policy /RSS'23/
14
78.3
92.5
68.3
11
50.5
11
72.4
Octo fine-tuned 23/
65
78.9
10
85.7
11
84.6
8
51.1
10
75.1
10
OpenVLA [CoRI'24)
60
84.7
88.4
10
79.2
9
53.7
9
76.5
To fine-tuned (RS
96.8
98.8
95.8
3
85.2
94.2
To-Past /RSS 2
96.4
96.8
7
60.2
85.5
To.5 111 [arXiv'25]
[8]
98.0
97.8
95.6
4
85.8
96.0
OpenVLA-OFT [RSS 29
97.6
98.4
3
97.9
1
94.5
2
97.1
SpatialVLA 25] 150
88.2
89.9
78.6
10
55.5
78.1
PD-VLA* /arXiv'25/
95.5
96.7
94.9
6
91.7
94.7
STAR /ICML'25 [23]
95.5
98.3
95.0
5
88.5
94.3
Dita (arXiv 225] [25]
84,2
96.3
8
85.4
9
63.8
82.4
CoT-VLA [5)
84
87.5
91.6
8
87.6
8
69.0
83.9
CogVLA
98.6
98.8
96.6
2
95.4
97.4
Table 2: Real-world Experimental Results. Performance comparison the Cobot Agilex ALOHA
tasks. "f" indicates our reproduced results, while "** denotes results reported the original paper.
Method
Object Placement
Drawer Manipulation
T-shirt Folding
Average
Cube-+Plate +Toy-Boww
Open +Place +Close
Step
+Step +Step
SR
VQ-BeT
3
5/10
3/10
4/10
3/10
1/10
20.0%
QueST
6/10
4/10
3/10
1/10
0/10
20.0%
STAR*
25
8/10
6/10
6/10
4/10
3/10
45.0%
PD-VLAT
62
8/10
7/10
6/10
6/10
4/10
7/10
6/10
4/10
50.0%
OpenVLA-OFT%
20
8/10
7/10
8/10
6/10
5/10
7/10
7/10
5/10
56.7%
VLg
9/10
8/10
8/10
7/10
7/10
9/10
8/10
6/10
70.0%
Bidirectional Action Chunk Decoding. To support coherent action generation, bidirectional attention employed within the action decoding, allowing full context integration among future action tokens:
Att (Ar) Softmax AA.A M A:
(18)
where M E ( ( D) enables full continuous parallel decoding within each action chunk while maintaining causal dependencies from risual-languaan inputs.
Unified Hybrid Attention Mask. A global attention mask M E RIM+T+K x (D)+
enforces hierarchical token dependencies across vision, language, and action modalities:
CAtten(X)) Softmax
X
Mhybrid
X
(19)
Vd
VL
causal
00
00
Mnybrid
0
0
(20)
0
0
M act
bi
This coupled attention structure enables CogVLA to retain fine- fine-grained cross-modal alignment and planning consistency under significant sparsification of visual inputs, ensuring that action sequences remain both semantically relevant and temporally coherent throughout the decoding process.
3 Experiments
3.1 Experiments Setting
All experiments are conducted on 4x A800 GPUs (80GB), benefiting from CogVLA's efficient instruction-driven sparsification. Implementation details in Appendix A.
Simulation Benchmark. We use the LIBERO benchmark 41 to evaluate task performance and efficiency. Its long and diverse instructions (avg. 10.48 words vs. 3.34 in RLBench) reflect the

---

Table 3: Efficiency Optimization Results. CogVLA maintains superior performance while achieving the highest efficiency. Ablation studies on Stage and Stage 2 validate the efficiency contribution of each routing module. "f" indicates our reproduced results.
Method
Inference Time
Throughput
FLOPs 4
Taining Cost +
LIBERO SR 1
Open VLA
30
0.254
3.9 Hz
8,48 T
11.7 h/10k steps
76.5%
OpenVLA-OOFT
0,132
60.6 Hz
8.45 T
12.5 h/10k steps
97.1%
PD-VLAf
0.143
55.9
8.48 T
11.7 h/10k steps
94.7%
Vog
0.091
87.9 Hz
2.72 T
4.7 h/10k steps
97.4%
w/o Stage
0.162
49.4
5.38
8.4 h/10k steps
w/o Stage 2
0.117
68.4 Hz
3.52 T
5.3 h/10k steps
Put both the cream cheese box and the butter the basket.
CogVLA: Faster and Hore Accurate
OFT
OpenVLA-OFT
LIBERO
pick
Open the drawer, place the toy the drawer, and then close It.
start
open
grasp
put
grasp
close
collision
close
Figure 4: Visualization comparison between CogVLA and OpenVLA-OPP CogVLA outperforms OpenVLA-OFF in success rates on both simulation and real-world tasks, achieving tfte-of-the-ar performance with 31% % reduction in inference time. It also demonstrates superior training efficiency, requiring 3.1 fewer FLOPSs and 2.7x shorter training time.
model's language understanding. LIBERO covers four suites suites-Spaatee Sppaiaal,, Object, Goal, and Long each with 10 tasks and 50 demonstrations.
Real-World Experiments. VLg is deployed on the Cobot Agilex ALOHA platform for three long-horizon tasks: Object Placement, Drawer Manipulation, and T-shirt Folding (45, 45, and 30 demonstrations). We introduce spatial and semantic variations during data collection.
Baselines. We compare CogVLA with multiple tlate-ofthhe-a methods, such OpenVLA, Spa- tialVLA, STAR, and CoT-VLA. For efficiency assessment, we further evaluate OpenVLA, along with the top-performing PD-VLA and OLenVLA-OFT- improved variant of OpenVLA that achieves higher performance and efficiency- under the same tin-ttnnng and inference settings as CogVLA.
3.2 Performance improvement
Simulation Experimental Results. The diverse task suites in the LIBERO benchmark reflect varying levels instruction-followingg requirements from different perspectives. We conducted 500 trials for each task suite, and CogVLA achieved the highest success rate of 97. 74,, as shown in

---

Table Ablation study on model components. Prun- ing and TG-Pruning denote LFP- LFP-Routing w/o and w/ instruction guidance. All ablations maintain a fixed 8x overall sparsification ratio.
Table Ablation on sparsification ratio allocation. Spf.Ratio denotes sparsifica- tion ratio, which can be tuned based on performance-efficien trade-off.
Stage I Step 1 Step 2 Pruning TG-Pruning
Stage 2
Stage
Spatial SR
Stage
Stage 2 Spf.Ratio) Spatial SR
8
91.2 (-7.4)
91.2 (-7.4)
X
X
8 x
92.0 (-6.6)
96.0 (-2.6)
8
94.6 (-4.0)
/
95.2 (-3.4)
4 x
2 x
8 x 98.6
/
92.0 (-6.6)
Table 6: Comparison of Stage 1+2 with
/
96.2 (-2.4)
other visual compression methods.
J
92.0 (-6.6)
FastV
12
SliME 83
Stage 1+2
/
98.6
SR 88.2 (-10.4) 77.6 (-21.0)
98.6
Tab. This strong performance stems from CogVLA's 3-stage progressive design, which enhances instruction-driven perception throughout the manipulation process. Notably, CogVLA ranks second only in the LIBERO-Goal suite, primarily due deliberate trade-off between performance and reduces visual input by 8x compared to other VLA models in the table.
Real-world Experimental Results. We conducted real-world training and evaluation of the top four models from the LIBERO simulation benchmark on complex Long-horizon tasks with rich instructions (Object Placement and Drawer Manipulation) and the representative dual-arm task T-shirt Folding. As shown in Tab. 2 CogVLA achieved the highest subtask and overall success rates. To better assess instraction-followee data and applied data augmentation following LIBERO's protocol, including variations in spatial arrangements, manipulated objects, and their attributes. The results demonstrate that Cog VLL's performance advantage generalizes effectively to real-world tasks.
ability in real-world settings, collected ALOHA-based experimental
3.3 Efficiency Optimization
As shown in Tab. CogVLA achieves 2.79 faster inference time, 22.54 higher throughput, 3.12x lower FLOPs, and 2.49 reduction in training cost compared to OpenVLA Moreover, CogVLA also outperforms state-of-the-an efficient VIAA models such as OpenVLA-OFF and PD-VL.A both training and inference efficiency. These gains stem from: 1) instruction- driven vision sparsification achieved via EFA- EFA-Routing and LFP-Routing, reducing visual input by up to 8x; and 2) parallel action decoding enabled by bidirectional attention in the CAtten module.
3.4 Qualitative Analysis
As shown in Fig. we compare the performance and efficiency of Cog and OpenVLA-OFT in both the LIBERO simulation and the ALOHA real-world setting. With strong instruction guidance and improved logical consistency, Cog executes manipulation tasks more accurately and avoids failures such drawer collisions (e.g., row 3, column 5). Cog VLA also demonstrates shorter action inference time, with its efficiency advantage becoming more pronounced as the task length increases.
3.5 Ablation Studies
Tab. validates the effectiveness of each module within CogVLA's 3-stage progressive design, highlighting their synergistic contributions under unified framework. Tab. 5 presents different sparsity ratio allocations across Stage 1 and Stage 2 under fixed 8x vision sparsification. Results show that both stages contribute to performance gains, with larger Stage ratios yielding greater improvements, as Stage further filters instruction-relevant tokens based on Stage outputs. Tab. demonstrates that the combined Stage 1+2 sparsification outperforms existing methods, as it is more instruction-drives and deeply integrated into the overall Cog VLA architecture. Additional ablation studies are provided in Appendix C.3.

---

Related Work
Vision-Language Action (VLA) Models. Vision-Language Models (VLMs)
35
11 42 22
57
79
[1
37
68
63
86 have propelled robotic control by providing rich multimodal representations, 70
fostering the development of VLA models [58
76 6 that bridge perception and action generation. Early works like CLIPort [59 and PerAct [60] aligned visual features with
43
75
67
56
introduced action tokenization to enable scalable web-to-robot transfer. More recently, Octo 665 constructed diverse multi-robot dataset to support multitask training, while OLe (30 demonstrated superior generalization to household proposed heterogeneous co-training across robots and semantic prediction tasks to enhance open-world generalization. However, directly fine-tuning billion-parann cillion-prrameter VLMs for action prediction remains computationally intensive, limiting scalability.
action policies. The RT series 9 g
tasks compared to diffusion-based methods. The π series [6 27
Efficient Design in VLA Models. Improving VLA efficiency has largely followed two paths: LLM- centric and vision-centric. LLM--cetricc approaches include Mixture-of- Depth (MoD) pruning [1] dynamic reasoning depth adjustment [74 [82) sparse Mixture-of-Expere (MoE) architectures
46
16
80) and lightweight models like DeeR-VLA 74,, RoboMamb [44,, and VLn
all aiming reduce decoding overhead. Vision-centric approaches focus on reducing the number of visual tokens passed to the I.M,, employing techniques such as patch token selection techniques [45] 26),, and an additional compression module [3 33 73 However, naively adapting these methods often leads to semantic inconsistency across modalities due to a lack of unified sparsification. To address this, we propose a cognition- aligned, instruction-driven sparsification framework that jointly improves efficiency and cross-modae consistency.
69
based on similarity
[55] 38), cropping-base
5 Conclusion
We presented CogVLA, a cumnitioonallinned and instruction-drivs Vision-Language work designed to address the computational inefficiencies and semantic fragmentation in existing VLA models. By integrating EFA-Routing, LFP-RRuting, and CAtten into a unified 3-stage progres- sive design, Vog achieves effective vision sparsification and coherent eross-modal reasoning. Extensive evaluations on both the LIBERO benchmark and real-world robotic tasks demonstrate that CogVLA not only achieves state-of-the-art performance but also significantly reduces computational cost and inference latency. This work highlights the importance of instruction-driven multimodal sparsification in building scalable and efficient embodied AI systems.
frame-
Acknowledgement
This study is supported by National Natural Science Foundation of China (Grant No. 62306090), Natural Science Foundation of Guangdong Province of China (Grant No. 2024A1515010147) and Shenzhen Science and Technology Program (KQTD22924022910220000)
References
[1] Dean-Baptiste Alayrac, Jeff Donahue, Pauline Luc. Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch. Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23733 2022.
[2] Bai, S Bai, S Yang, S Wang. S Tan, P Wang, J Lin, C Zhou, and Zhou. Qwen-vl: A fronticr large vision-language model with versatile abilitics. arxiv 2023. arXiv preprint arXiv:2308. 12966.
[3] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile lasion-l model for understanding, localization, text reading. and beyond. arXiv preprint arXiv:2308. 12966, 2023.
[4] Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quon Vuong, Jonathan Tompson, Yevgen Chebotar, Debidatta Dwibedi, and Dorsa Sadigh. Rt-h: Action hierarchies using language. arXiv preprint 20Xiv:2403001823 2024.
10

---

[5] Homanga Bharadhwaj. Jay Vakil, Mohit Sharma, Abhinav Gupta. Shubham Tulsiani, and Vikash Kumar. Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking. 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 4788-4795. IEEE, 2024.
[6] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. Ro: A saiion-anguage-actinnn flow model for general robot control, 2024. URL https://arxiv. 0rg/abx/2410.24164
[7] Neil R Bramley, Tobias Gerstenberg, Joshua B Tenenbaum, and Todd M Gureckis. Intuitive experimentation in the physical world. Cognitive psychology, 105:9-38, 2018.
[8] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-actioo models transfer web knowledge robotic control. arXiv preprint arXiv:2307.188// 2023.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022.
[10] Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06/55 2024.
[11] Gongwei Chen, Leyang Shen, Rui Shao, Xiang Deng, and Liqiang Nic. Lion: Empowering multimodal large language model with dual-level visual knowledge. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 26540-26550 2024.
[12] Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang, An image is worth 1/2 tokens after layer 2: Plug-and and-play inference acceleration large vision-languse models. In European Conference on Computer Vision, pages 19 39-35. Springer. 2024.
[13] Zhixuan Chen, Xing Hu, Dawei Yang. Zukang Xu, Zhihang Yuan, Sifan Zhou, al. Moequant: Enhancing quantization for mixture-of-expert large language models via expert-balanced sampling and affinity guidance. In Forty-second International Conference Machine Learning.
[14] Cheng Chi, Zhenjia Xu, Siyuan Feng, Fric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 027836492411736665 2023.
[15] Maurizio Corbetta and Gordon Shulman. Control of goal-directed and cimuulusddriven attention in the brain. Nature reviews neuroscience, 3(3):001 215, 2002.
[16] Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Cao, Deli Chen, Jiashi Li, Wangding Zeng. Xingkai Yu, Wu. et al. Deepseekmoe: Towards ultimate expert specialization mixture-of-experts language models. arXiv preprini 240.::22000000666, 2024.
[17] Luciano Del Corro, Allie Del Giomo, Sahaj Agarwal, Bin Yu, Ahmed Awadallah, and Subhabrata Mukherjee. Skipdecode: Autoregressive skip decoding with batching and caching for efficient Ilm inference. arXiv preprint arXiv:2307.02628. 2023.
[18] Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, et al. Knowledge insulating vision-language-ace models:
Train fast, fast, generalize better. arXiv preprint arXiv:2505.23700 2025.
[19] Leonardo Fogassi, Pier Francesco Ferrari, Benno Gesierich, Stefano Rozzi, Fabian Chersi, and Giacomo Rizzolatti. Parietal lobe: from action organization to intention understanding. Science, 308(5722):662 667, 2005.
[20] Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. In Conference on Robot Learning (CoRL), 2024.
[21] Vittorio Gallese, Luciano Fadiga, Leonardo Fogassi, and Giacomo Rizzolatti. Action recognition in the premotor cortex. Brain, 1 11(()):53366) -09, 1996.
[22] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
11

---

[23] Li Hao. LV Qi. Shao Rui, Deng Xiang, Li Yinchuan, HAO Jianye, and Nie Liqiang, Star: Learning diverse robot skill abstractions through rolation-augmeented vector quantization. International Conference on Machine Learning (ICML), 2025.
[24] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), arXiv preprint arXiv:1606.08415 2016.
[25] Zhi Hou, Tianyi Zhang, Yuwen Xiong, Haonan Duan, Hengjun Pu, Ronglei Tong, Chengyang Zhao, Xizhou Zhu, Yu Qiao, Jifeng Dai, al. Dita: Scaling diffusion transformer for generalist vision-language-action policy. arXiv preprint arXiv:2503. 19757, 2025.
[26] Runhui Huang. Xinpeng Ding, Chunwei Wang, Jianhua Han, Yulong Liu, Hengshuang Zhao, Hang Xu, Hou, Wei Zhang, and Xiaodan Liang. Hires-llava: Restoring fragmentation input in high-resolution large vision-language models. arXiv preprint 2rXXiv2407.08706 2024.
[27] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. mo.s: a Vision-langunge-action model with open-world generalization. arXiv preprint arXiv:2504. 250444 2025.
[28] Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J. Davison. Rlbench: The robot leamning benchmark & learning environment. IEEE Robotics and Automation Letters, 2020.
[29] Moo Jin Kim, Chelsea Finn, and Perey Liang. Fine-tuning vision-languago-actiee models: Optimizing speed and success. arXiv preprint arXiv:2502. 19645, 2025.
[30] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source assionlanguagg-actiinn model. arXiv preprint 2iw.:24406092444040 2024.
[31] Moo Jin Kim, Karl Pertsch. Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong. Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model. In 8th Annual Conference on Robot Learning, 2024.
[32] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253, 2017.
[33] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253, 2017.
[34] Seungjae Lee, Yibin Wang, Haritheja Etukuru, H Jin Kim, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Behavior generation with latent actions. arXiv preprint arXiv:2403.03181, 2024.
[35] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2lip-2: Bootstrapping language- -magg pre-training with frozen image encoders and large language models. In International conference machine learning, pages 19730 19742. PMLR, 2023.
[36] Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational Sasion-languag--cttion model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:241
2024.
[37] Wei Li, Bing Hu, Rui Shao, Leyang Shen, and Liqiang Nie. Lion-fs: Fast & slow video-language thinker as online video assistant. In Proceedings of the Computer Vision and Puttern Recognition Conference. pages 224003251. 2025.
[38] Wentong Li, Yugian Yuan, Jian Liu, Dongqi Tang, Song Wang, Qin, Jianke Zhu, and Lei Zhang Tokenpacker: Efficient visual projector for multimodal Ilm. arXiv preprint arXiv:2407.0225 2024.
[39] Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang. Yabo Sun, Yuliang Liu, and Xiang Bai. Monkey: Image resolution and text label are important things for large multi-modal models, In pruceedings of the IEEE/CVF conference on computer vision and pattem recognition, pages 26763-26773, 2024.
[40] Bin Lin, Zhenyu Tang. Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Jinfa Huang, Junwu Zhang, Yatian Pang. Munan Ning, et al. Moe-llava: Mixture of experts for large vision-languee models. arXiv preprint arXiv:2401.1 15947, 2024.
[41] Bo Liu, Yifeng Zhu Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, arXiv preprint 230::2306.0330000 2023.
12

---

[42] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296-26306, 2024.
[43] Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang. Chenyang Gu, Xiaoqi Li, Ziyu Guo. Sixiang Chen, Mengzhen Liu, et al. Hybridvla: Collaborative diffusion and autoregression in a unified vision- waa.-aaa-agguttiionoo model. arXiv preprint arXiv:2503. 10631, 2025.
[44] Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Lily Lee, Kaichen Zhou, Pengiu An, Senqiao Yang, Renrui Zhang. Yandong Guo, and Shanghang Zhang. Robomamba: Multimodal state space model for efficient robot reasoning and manipulation. arXiv preprint wrXiv:2406.43332 2024.
[45] Yuliang Liu, Biao Yang. Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, and Xiang Bai. Textmonkey: An ocr-free large multimodal model for understanding document. arXiv preprint arXiv:2403.04473 2024.
[46] Yaxin Luo, Gen Luo, Jiayi Ji, Yiyi Zhou, Xiaoshuai Sun, Zhiqiang Shen, and Rongrong Ji. y mod: Explor- ing ofxture-of-ddpth adaptation for multimodal large language models. arXiv preprint arXi::2400.13899 2024.
[47] Atharva Mete, Haotian Xue, Albert Wilcox, Yongxin Chen, and Animesh Garg. Quest: supervuud skill abstractions for learning continuous control, 2024.
[48] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
[49] Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision- siioo-langgage-acee action models. arXiv preprint W7X1v:250.00747, 2025.
[50] Delin Qu, Haoming Song. Qizhi Chen, Yuanqi Yao, Xinyi Ye. Yan Ding, Zhigang Wang, Yua Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for arXiv preprint arXiv:2501.15830 2025.
model.
[51] David Raposo, Sam Ritter, Blake Richards, Timothy Lillicrap, Peter Conway Humphreys, and Adam Santoro, Mixture-of-depths: Dynamically allocating compute transformer-hased language models. arXiv preprint Xr 2024.
[52] Luca M Schulze Buschoff, Elif Akata, Matthias Bethge, and Eric Schulz. Visual cognition in multimodal large language models. Nature Machine Intelligence, pages 1-11, 2025.
[53] Luca M Schulze Buschoff, Elif Akata, Matthias Bethge, and Eric Schulz. Visual cognition in multimodal large language models. Nature Machine Intelligence, pages 1-11, 2025.
[54] Michael Schwartze, Kathrin Rothermich. and Sonja A Kotz. Functional dissociation pre-sma and sma-proper in temporal processing. Neuroimage, 60( (0(1):290-299) 298, 2012.
[55] Yuzhang Shang. Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. arXiv preprint arXiv:2403. 1..311555 15388, 2024.
[56] Rui Shao, Wei Li, Lingsen Zhang, Renshan Zhang. Zhiyang Liu, Ran Chen, and Liqiang Nie. Large vlm- based lisgua-la acctinn models for robotic manipulation: A survey, arXiv preprint arXiv:2508.1307. 2025.
[57] Leyang Shen, Gongwei Chen, Rui Shao, Weili Guan, and Liqiang Nie Mome: Mixture of multimodal experts for generalist multimodal large language models. Advances in neural information processing systems, 42:000 2024.
[58] Lucy Xiaoyang Shi, Brian Ichter, Michael Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, et al. Hi rohot: Open-ended instruction following with hierarchical vision-language action models. arXiv preprint urXiv:2502.19417, 2025.
[59] Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipula- tion. In Conference on robot learning, pages 894-906. PMLR, 2022.
[60] Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A mulli--ask transformer for robotic manipulation. In Conference Robot Learning, pages 785-799, PMLR, 2023.
13

---

[61] Kevin A Smith, Peter W Battaglia, and Edward Vul. Different physical intuitions exist between tasks, not domains. Computational Brain & Behavior. 1:101-118, 2018
[62] Wenxuan Song, Jiayi Chen, Pengxiang Ding, Han Zhao, Wei Zhao, Zhide Zhong, Zongyuan Ge, Jun Ma, and Haoang Li. Accelerating aasion-language-ctiann model integrated with action chunking via parallel decoding. arXiv preprint arXiv:2503.02310, 2025.
[63] Yu Sun, Yin Li, Ruixiao Sun, Chunhui Liu, Fangming Zhou, Ze Jin, Linjie Wang. Xiang Shen, Zhuolin Hao. and Hongyu Xiong, Audio-enhanced vision- -siiunlaagguage modeling with latent space broadening for high quality data expansion, 2025.
[64] Shoji Tanaka and Eiji Kirino. Dynamic reconfiguration of the supplementary motor area network during imagined music performance. Frontiers in human neuroscience, 11:606, 2017.
[65] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint 240:: 12213, 2024.
[66] Joshua Tenenbaum, Charles Kemp, Thomas Griffiths, and Noah D Goodman. How to grow a mind: Statistics, structure, and abstraction. science, 331 (6022)))27791188 2011.
[67] Sicheng Wang. Sheng Liu, Weiheng Wang. Jianhua Shan, and Bin Fang. Robobert: An end-to-end multimodal robotic manipulation model. arXiv preprint trXiv:2502.07837, 2025.
[68] Zixuan Wang, Yu Sun, Hongwei Wang, Baoyu Jing, Xiang Shen, Xin Dong, Zhuolin Hao, Hongyu Xiong, and Yang Song. Reasoning-enhancedd domain-adapiion pretraining of multimodal large language models for short video content moderation, 2025.
[69] Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Zhibin Tang, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, et al. Tinvlla Towards fast, data-efficient vision-languaae-accioon models for robotic manipulation. IEEE Robotics and Automation Letters, 2025.
[70] Rosa Wolf, Yitian Shi, Sheng Liu, and Rania Rayyes. Diffusion models for robotic manipulation: A survey. arXiv preprint ar arxivv2504.00000 25444880 2025.
[71] Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation. arXiv preprint 2rXiv:2312.133.99 2023.
[72] Shiwei Wu, Joya Chen, Kevin Qinghong Lin, Qimeng Wang, Yan Gao, Qianli Xu, Tong Xu, Yao Hu, Enhong Chen, and Mike Zheng Shou. Videollm-mod: Efficient video-language streaming with mixture- of-depths vision computation. Advances in Neural Information Processing Systems, 77:109922-1199477 2024.
[73] Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug-ow12. Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 13000-33511 2024.
[74] Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang. Shiji Song, Jiashi Feng, and Gao Huang. Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution. Advances in Neural Information Processing Systems, 1966619-5663, 2024.
[75] Shuang Zeng. Xinyuan Chang, Mengwei Xie, Xinran Liu, Yifan Bai, Zheng Pan, Mu Xu, and Xing Wei. Futuresightdrive: Thinking visually with 1785 2025.
cot for autonomous driving. arXiv preprint
[76] Shuang Zeng. Dekang Qi, Xinyuan Chang, Feng Xiong, Shichao Xie, Xiaolong Wu, Shiyi Liang, Mu Xu, and Xing Wei. Janusvin: Decoupling semantics and spatiality with dual implicit memory for vision- language navigation. arXiv preprint arXiv:2509. 22548, 2025.
177] Jun Zhang, Desen Meng, Ji Qi, Zhenpeng Huang, Tao Wu, and Limin Wang. p-mod: Building mfxture-of- depths milms via progressive ratio decay. arXiv preprint arX1::4412.04449, 2024.
[78] Jusheng Zhang, Kaitong Cai, Yijia Fan, Jian Wang, and Keze Wang. CF-VIm.coumterfaccual lision-language fine-tuning, 2025.
[79] Jusheng Zhang, Yijia Fan, Weajun Lin, Ruiqi Chen, Haoyi Jiang, Wenhao Chai, Jian Wang, and Keze Wang. Gam-agent theoretic and uncertainty- aware collaboration for complex visual reasoning. arXiv preprint 250::2500523999 2025.
14

---

[80] Jusheng Zhang, Zimeng Huang. Yijia Fan, Ningyuan Liu, Mingyan Li. Zhuojie Yang, Jiawei Yao, Jian Wang, and Keze Wang. Kabb: Knowledge-aware bayesian bandits for dynamic expert coordination in multi-agent systems, 2025.
[81] Renshan Zhang. Rui Shao, Gongwei Chen, Miao Zhang, Kaiwen Zhou, Weili Guan, and Liqiang Nie. Falcon: Resolving visual redundancy and fragmentation high-resolution multimodal large language models via visual registers. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), October 2025.
[82] Rongyu Zhang, Menghang Dong, Yuan Zhang, Liang Heng, Xiaowei Chi, Gaole Dai, Li Du, Dan Wang. Yuan Du, and Shanghang Zhang. Mole-vla: Dynamic layer-skipping vision language action model via mixture-of-layers for efficient robot manipulation. arXiv preprint arXiv:2503.20:: 20384, 2025.
[83]
Yi-Fan Zhang, Qingsong Wen, Chaoyou Fu, Xue Wang, Zhang Zhang. Liang Wang. and Rong Jin. Beyond Ilava-hd: Diving into high-resolution large multimodal models. arXiv preprint 2rXiv:2406.0884 2024.
[84] Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang 7hang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of- reasoning for visionlanguage-accinn models. arXiv preprint arXiv:2503.22020 2025.
[85] Yifan Zhong, Xuchuan Huang, Ruochong Li, Ceyao Zhang, Yitao Liang, Yaodong Yang. and Yuanpei Chen. Dexgraspvla: A vision-language-attoo framework towards general dexterous grasping. 2025.
[86] Sifan Zhou, Shuo Wang, Zhihang Yuan, Mingjia Shi, Yuzhang Shang, and Dawei Yang. GSQ-tuning: Group-shared exponents integer in fully quantized training for LLMs on-device fine-tuning In Findings of the Association for Computational Linguisties: ACL 2025, pages 22971 -22988, Vienna, Austria, July 2025. Association for Computational Linguistics.
[87] Zhiyuan Zhou, Pranav Atreya, You Liang Tan, Karl Pertsch, and Sergey Levine. Autoeval: Autonomous evaluation of generalist robot manipulation policies in the real world, arXiv preprint arXiv:2503.24277 2025.
[88] Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Weibin Meng, Ran Cheng, Yaxin Peng, Chaomin Shen, and Feifei Feng. Chatvla: Unified multimodal understanding and rohot control with vision lision-lagguggeactton model, 2025.
NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: We state our contributions in the abstract and at the end of the introduction, while the scope is outlined in the abstract and the beginning of the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We mentioned our limitations in Appendix.
Guidelines:
15

---

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This paper does not involve the formulation of theorems.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide implementation details in Appendix ensure the reproducibility of the experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
16

---

If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We train and evaluate method using open-sourced datasets and models, which facilitates easy replication. Following acceptance, both the code and the data will be made available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. ec/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /nips.cc/public/gu.dee
wwwpabnisssoooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
17

---

6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We detail our experimental setting in Section 3 and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report experimental results under different random seed settings in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We introduce this in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
18

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We do not engage in research that violates ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We describe the potential impacts in Appendix.
Guidelines:
The answer NA means that there no socictal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We describe the potential impacts in Appendix.
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We utilized open-source models and datasets in accordance with the license.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We develop comprehensive guidelines for running our code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing research with human subjects
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: The LLM is used solely for writing and grammatical refinement purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
21

---

CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification
Appendix
This appendix provides comprehensive supplementary material to support the methodology, analysis, and findings presented the main paper.
Section describes implementation details, including model and training details.
Section outlines experimental details for both simulation and real-world settings.
Section presents extended quantitative analyses, including multi-seed evaluations, additional
ablation studies, and expanded real-world results.
Section provides supplementary qualitative analyses, such as diverse task executions and
crrructiontt-oobservtiioo attention visualizations.
Section E limitations, and reflects on the broader societal implications and potential risks.
discusses additional insights into the motivation behind CogVLA, highlights its current
We provide third-person view videos at http://ttaaaaa.gggbbbooooCCCa giggtubbio/CooVVLAA-aa. CogVLA performing manipulation tasks in fully autonomous mode, played at Ix speed. Due to the requirement of remote communication during each action chunk prediction, slight delays are introduced by network latency. For future deployments, plan to run CogVLA locally on hardware with more than 20 GB of GPU memory (e.g., RTX 4090 with 24 GB) to eliminate such latency.
demonstrating
A Implementation Details
A.1 Model Details
EFA-Routing. In Step 1, each of the two vision encoders uses 64 aggregation tokens, thereby reducing the number of visual tokens to 25% of the original. In addition, the scale and shift vectors for FiLM, " and Bi, are derived from linear transformation of the text embedding. In Step a two-layer MLP is applied to the text embedding to produce routing weights for the two vision encoders.
LFP-Routing. this module, we employ a shifted cosine schedule 77 to control the proportion of visual tokens retained at each layer. The formulation is as follows:
al cos +1, 1= 1,2, .........
B:
(21)
where L denotes the total number layers in the LLM, which is L == 32 for CogVLA. The constant 77 is shift factor that vertically adjusts the cosine decay curve, providing flexible mechanism to control the overall computational cost of the model. In our implementation, 7) is set to 0.5. Specifically, apply clamp operation constrain 83) within the range (0.05, 0.85). As result, LFP-Routing achieves approximately a 50% token pruning rate.
In addition, the instruction-conddtioneed scaling and shifting functions (-)(()) and PLIM(.) in LFP- Routing are both implemented using two-layer MLPs, with a hidden layer dimension of 2048, resulting in parameter count almost identical to that of a direct linear layer.
A.2 Training Details
LIBERO Training Setup. We adopt OpenVLA [1
the backbone model and set the action chunk
size K 8. Fine-tuning is performed using Low-Rank Adaptation (LoRA) with a rank of and an value of 64. The model is trained for 60K steps with batch size of 64 and an initial learning rate of 5e-4. Checkpoints are evaluated every 10K steps, and the best-performing checkpoint is selected for reporting.
22

---

Real-World Training Setup. For the real-world experiments, we set the chunk size to K = 25 and fine-tune VLe using LoRA with a rank of 32 and an alpha value of 64. The model is trained with a batch size of 32 for total of 80K steps. The initial learning rate was set to 5e-4, which is reduced to 5e-5 after 50K steps. Starting from step 60K, we evaluate checkpoints every 10K steps and report the best-performing checkpoint.
B Experimental Details
B.1 Simulation Benchmark
We evaluate CogVLA on the LIBERO simulation benchmark [4]] standardized suite of language- conditioned robotic manipulation tasks. Unlike earlier benchmarks such as RLBBnch [28). LIBERO features more complex and diverse instructions, averaging 10.48 words per command compared to
only 3.34 in RLBench. This makes it more suitable testbed for assessing the model's s capacity in language grounding and multimodal reasoning. LIBERO comprises four task suites-Spatian Object, Goal, and Long-each containing 10 tasks with 50 bulap-eeleoperated demonstrations. These suites are designed to probe distinct reasoning capabilities:
LIBERO-Spatial evaluates spatial reasoning capabilities by presenting identical objects arranged in different spatial configurations. The agent must interpret spatial relations (e.g.. left/right, front/behind) described in the instruction to complete the task correctly.
LIBERO-Object measures the model's ability to generalize across object categories. While spatial layouts remain fixed, the manipulated objects vary in type, shape, or color, requiring the agent to ground object-referential language and adapt its actions accordingly.
LIBEERO-Goal tests task-oriented comprehension by altering the goal specification while keeping object types and spatial layouts constant. The agent must disambiguate subtle differences in instruction semantics to execute distinct manipulation outcomes.
LIBERO-Long challenges the agent with multi-step. long-horizon tasks involving diverse objects and environments. Success requires not only grounded perception and instruction following, but also sequential planning.
CogVLA is trained and evaluated under the same setting as Open VLA 33 to ensure comparability, We report results on all four suites to validate the model's generalization, efficiency, and semantic grounding capabilities.
B.2 Real-World Setup
We deploy Cog on Cobot Agilex ALOHA 20 manipulation platform, to validate its world applicability. The coordinated dual-arm manipulation. To assess robustness and generalization, we introduce moderate data augmentation by varying object attributes (c.g., size, color) and rearranging spatial layouts.
evaluation consists of five diverse tasks involving both single-arm and
We collect real-world training data for the Cobot Agilex ALOHA robot via human teleoperation. For Tasks 1 1-5, we gather 45, 45, 30, 30, and 45 expert demonstrations, respectively. We report the results of Tasks 1-3 in the main paper, and provide additional results for Tasks 4-5 in this appendix. The instructions and descriptions for Tasks 1-5 are provided below:
Task 1: "Put the cube into the plate, and then put the toy into the bowl." A two-step pick-and-plan sequencing. This is dual-arm task consisting of two sequential subtasks: 1) "Put the cube into the plate" " with the left arm, and 1) "Pul the toy into the bowl" with the right arm. Task success achieved only when both subtasks are completed successfully. We report success rates for each subtask and the overall task.
task involving object category understanding and temporal
Task "Open the drawer, place the toy into the drawer, and then close it." A composite task requiring interaction with articulated objects and multi-stage execution. This a dual-arm task consisting of three sequential subtasks: 1) "Open the drawer" with the left arm, 2) "Place the toy into the drawer" with the right arm, and "Close the drawer" with the left arm. Task success requires three subtasks be completed. We report success rates for each subtask and the overall task.
23

---

Table Multi-seed evaluation results in simulation. Task success rates (SR) are compared across four task categories on the LIBERO benchmark. "I" denotes our reproduced results. CogVLA
demonstrates strong and consistent performance.
Method
Spatial SR t
Object SR 1
Goal SR f
Long SR
Average
SR
RK
OL /CoRL /4/
30
84.7 0.9
88.4 + 0.8
79.2 = 1.0
53.7 = 1.3
76.5
0.6
5
SpatialVLA /RSS'25]
50
88.2 0.5
89.9 + 0.7
78.6 = 0.6
55.5 = 1.0
78.1
0.7
STAR /JCML'25] [23]
95.5 0.6
98.3
0.2
95.0 + 0.7
88.5 + 0.3
94.3
0.1
2
CoT-VLA [CV
[84]
87.5 1.4
91.6
+ 0.5
87.6
+ 0.6
69.0
+ 0.8
83.9
+ 0.6
3
CogVLA
98.5 0.5
98.8
0.4
96.5
0.6
95.2 1.1
97.4
0.4
1
Task 3: "Fold the T-shirt."
A soft-body manipulation task that evaluates the system's ability handle deformable objects. This is a dual-arm task consisting of three sequential folding steps. Task success is determined by the successful execution of all three steps. We report intermediate success rates for each step and the overall task performance.
Task 4: "Pick the red cube into the plate, and then pick the big cube into the bowl." A multi-attribute grounding task requiring comprehension of both color and size references. This is a dual-arm task consisting of two sequential subtasks: 1) "Pick the red cube into the plate" with the left arm, and 2) "Pick the big cube into the bowl" with the right arm. Task success is achieved only when both subtasks are completed. We report success rates for each subtask and the overall task.
Task 5: "Pick the left cube into the plate."
A spatial reasoning task focusing on relative positioning and egocentric understanding. This is single-arm task consisting of one pick-and-ple action. We report the final task success rate.
Supplementary Quantitative Analysis
C.1 Multi-Seed Evaluation
To evaluate the statistical robustness and consistency of CLA's performance, we conduct multi- seed evaluations on the LIBERO benchmark. For each of the four task suites (Spatial, Object, Goal, and Long), rate along with the standard deviation.
experiments using three independent random seeds and report the mean success
As shown in Tab. 7 CogVLA exhibits consistently high performance across different seeds, with standard deviations ranging from 0.2% to 0.6%. This indicates stable learning behavior and further validates the strong generalization capability of CogVLA's three-stage citructioo-driven architecture across diverse task types.
C.2 Extended Real-World Task Results
In addition to the results reported in the main paper, we present the performance of CogVLA on Tasks 4 and 5, as shown in Tab.
Task 4 ("Pick the red cube into the plate, and then pick the big cube into the bowl") evaluates the model's 's ability to ground multi-attribute language and execute sequential actions. CogVLA achieves the highest success rates across both subtasks and the overall task, demonstrating strong compositional understanding of attribute references such as color and size.
Task 5 ("Pick the left cube into the plate") focuses on egocentric spatial reasoning, requiring precise interpretation of relative spatial references from the agent's visual perspective. CogVLA maintains high success rate this setting, indicating robust grounding of spatial concepts.
24

---

Table 8: Extended real-world results on Tasks 4-5, Performance comparison on the Cobot Agilex ALOHA tasks. "I" indicates our reproduced results.
Method
Task
Task Left Cube PPlate
Average SR
Red Cube->Plate
+ +Big Cube Bowl
PD-VLA 62
7/10
5/10
6/10
60.0%
OL OFFF (29
7/10
6/10
6/10
63.3%
CogVI.A
8/10
7/10
8/10
76.7 %
These results further validate Cog VLA's ability to generalize to real-world tasks that demand fine- grained language grounding and spatial understanding.
C.3 Extended Ablation Studies
We extend the sparsification analysis by evaluating additional Stage 1/Stage 2 configurations: 2x2 and 4x4, while keeping the total sparsification ratio fixed at 4x and 16x, respectively. These configurations are compared alongside the baseline 8x setting with different asymmetric allocations (e.g., 2x-4x and 4x-2x), allowing us systematically assess how the distribution of sparsity across stages impacts downstream performance.
As shown in Tab.3, the 2x2 setting provides favorable trade-off between performance and computational effi- ciency. In contrast, the 4x4 setting leads to slight degradation in perfor- mance, suggesting that excessive spar- sification across both stages may hin- der the preservation of task-relevant information.
Table 9: Supplementary ablation on sparsification ratio allocation. Spf.Ratio denotes the sparsification ratio, which can adjusted based on the performance-effiiien off. Cog VLA achieves better performance when a relatively higher sparsification ratio is allocated to Stage compared to Stage 2.
trade-
Stage 1
Stage 2
Spf.Ratio
Spatial SR
FLOPs
Interestingly, the asymmetric config- urations, particularly the 4x-2x setup, outperform their symmetric counter- parts, achieving the highest spatial success rate 98.6. This highlights the advantage of applying more ag-
2 x
2
4
96.4 (-2.2)
3.87 T
4 x
4 x
16 X
93.2 (-5.4)
2.30 T
2 x
4
94.6 (-4.0)
2.72 T
4 x
2
X
98.6
2.72T
gressive token reduction in Stage (EFA (EFA-Routing), where redundant visual tokens can be effectively
compressed via instruction-guided aggregation. Subsequently, Stage 2 (LFP-Routing) performs finer-grained token pruning in a context-aware manner within the language model, allowing for better preservation task-relevant information.
These findings support the core design principle of CogVLA: progressive sparsification with an asymmetric allocation tailored to the representational characteristies of each stage. By balancing early-stage compression and late-stage selectivity, the model achieves both computational efficiency and high task accuracy, reinforcing the importance of stage-aware sparsity scheduling in multimodal architectures.
D Supplementary Qualitative Analysis
D.1 Additional Visualizations of Simulation and Real-Worke Results
We present additional qualitative results from both simulation and real-world experiments to illustrate Co''LA's generalization and execution capabilities. As shown in Fig. 8 the model consistently com- pletes multi-step tasks across diverse environments, object configurations, and instruction variants.
In real-world tasks with varying instructions, CogVLA accurately interprets long-horizon commands and produces coherent action sequences. These examples further highlight the model's ability to maintain cross-modal consistency and temporal reasoning, well as its robustness in simulation- to-reality transfer. Fig. illlustratss the real-world manipulation workflows for Tasks 1-5. For Task 1, we provide multi-view observations from the Front Camera, Left Wrist Camera, and Right Wrist
25

---

Figure 5: Real-world Manipulation Workflows and Visualization for Tasks 1-5. Each task panel illustrates the initial setup and CogVLA's execution process based on the natural language instruction. For Task I, multi-view observations from the Front Camera, Left Wrist Camera, and Right Wrist Camera are provided to capture dual-arm coordination. For Tasks 2-5, representative frames from the Front Camera highlight key manipulation stages. These visualizations support interpretation of task complexity and grounding behavior.
Camera. For Tasks 2-5, only Front Camera observations are shown for clarity. In Fig. 6 we present third-person view demonstration of VLA performing manipulation task in the lab. The corresponding MP4 video file provided in the supplementary materials.
D.2 Instruction-to-Obserr
Attention Maps
To gain deeper insights into how CogVLA aligns language instructions with visual observations, we visualize the attention maps generated by the cross-modal attention modules. As shown in Fig, 7 the attention weights highlight task-relevant regions the input image.
These visualizations demonstrate that CogAL instruction-aware routing mechanisms effectively guide the perception module to attend to semantically meaningful areas, enabling robust visual grounding even in cluttered or ambiguous scenes.
E
Discussion
E.1
Supplementary Details on the Motivation
Cog VLA is motivated by the need to improve both computational efficiency and cross-modal semantic alignment in instructio-connitionn robotic systems. Its architectural design is informed by cognitive science research on how humans process language, perceive their environment, and execute actions in coherent and goal-directed manner.
Cognitive studies suggest that humans rely on structured inductive biases- termed "intuitive theorics"- interpret the world. including intuitive physics, causality, and theory of mind [33 66).
26

---

outraction the stoees plae the kes the then
ARveSTABe
Fel: UELENEN
FEls GOUNEERL
Frlp AnvesvanI
FeB: AEVEREOU
Figure 6: Third-person visualization of CogVLA performing a manipulation task. The corre- sponding video is provided in the supplementary materials. Gripper details are highlighted with red circles.
While recent multimodal large language models exhibit partial competence in these areas, they often lack robustness in compositional reasoning and causally grounded behavior [53]
To address these limitations, CogVLA adopts a biologically inspired architecture that reflects the division of functional roles observed in the human brain. Specifically, we draw connections between the model's three routing modules and key components in human multimodal cognition: the Visual Attention System (VAS), the Supplementary Motor Area (SMA), and the Premotor Cortex (PMC).
Visual Attention System (VAS) > Farcode-FiiLM. The human visual attention system selectively enhances perception of task-relevant features while suppressing distractors [1].. Top-down signals from frontal and parietal cortices bias visual processing toward objects regions mentioned in language or necessary for action. This selective modulation improves efficiency and semantic grounding in complex scenes. In VLA, the Encoder-FiLM module mimics VAS by dynamically modulating visual encoder features conditioned on instructions, focusing perception on semantically relevant regions and reducing redundancy (48). This allows the model's perception be grounded in context, much as the brain's ' attention system tunes visual processing to relevant aspects of a scene during coordinated vision-language tasks.
Supplementary Motor Area (SMA) - LLM-FILM. The SMA plays a key role in planning and sequencing actions, even in the absence of physical movement (54 64]. It integrates multimodal information and high-level goals to shape future motor behavior, before engaging primary motor circuits. In CogVLA the LLM-FiLM module and serves a similar function: it injects task-specific intent into the language model, pruning irrelevant visual-ling lingul-ingguisic tokens and steering the model toward generating appropriate action plans. This enables more efficient and intention-alignen reasoning, analogous to how the SMA organizes abstract motor programs before execution.
be seen as the "intention planner" of the model
Premotor Cortex (PMC) - V-L-A Coupled Attention. The premotor cortex is involved in translating perceptual cues into executable motor plans [9]. It contains visuomotor neurons that represent both the perception of object affordances and the intended grasping actions, enabling visuomotor grounding. CogVLA's V-L-A Coupled Attention module reflects this mechanism by integrating visual, linguistic, and action representations through a unified attention mechanism. This ensures that generated actions are causally and temporally coherent with respect to both the observed scene and the given instruction.
By aligning its modular design with biologically plausible cognitive functions, CLg offers not only performance and efficiency gains, but also cognitively grounded pathway for improving generalization and interpretability in embodied multimodal agents.
27

---

Instruction: Pick black
ramekin and penspeeiive
ZWHO22 wist mounted camara perspective
Sitlp maln camara perspcttvve
Figure 7: Attention maps between aggregation tokens and patch tokens in DINOv2 and SigLIP. We visualize the attention maps from 17 out of 64 aggregation tokens to the patch tokens of the observation, covering four sets of visualizations across two visual encoders and two camera views. The input language instruction is: "Pick up the black bowl between the plate and the ramekin and place it the plate." Both DINOv2 and SigLIP exhibit varying degrees of focused attention patch tokens relevant to the instruction.
E.2 Limitation and Future Work
While CLg demonstrates strong performance across simulation and real-world tasks, several limitations remain. First, the current instruction-to-vinne routing relies on predefined sparsity ratios and fixed token pruning schedules, which may not adapt optimally varying instruction complcxity or scene difficulty. Second, although the model generalizes well within the LIBERO and ALOHA settings, its performance under out-of-distribution instructions unseen manipulation categories is yet to be thoroughly evaluated.
In future work, we aim to explore adaptive sparsification mechanisms conditioned task semantics and environmental uncertainty. Moreover, integrating lifelong learning or online adaptation strategies may further enhance CogVLA's robustness in open-world deployment scenarios. Lastly, extending the framework to support multimodal feedback (e.g., haptic or force sensing) could improve its applicability to fine-grad fine-grained manipulation tasks.
28

---

LIBERO-Spatial Spetial pick aa, black, Ltt, deawer, of wooden, cabinet and place plate
eIpeRD objict, pick ap. the beq Muce, and place, Rt in, the, dasket
LIBERO object pick, the aquhabet soup. and piace. Uo.. the Oaskee
LIBERO-GOni bottie. rack
Long SCENE6_ jaCt ywllow and Aicronove, and
LIBERO-long: LIMNG, ROON, SCENCS put the white, mag. plate, and put yellow, mag on the right plate
LIBFRG-Cong: STUDY SCEUEL pice the book and place in the back compartment of the caddy
Figure 8: Manipulation Workflows and Visualizations in the LIBERO Simulation Environment. We present the execution processes CogVLA across LIBERO-Spatial, LIBERO-Object, LIBERO- Goal, and LIBERO-Long demonstrating its strong performance under diverse instructions and wide range of tasks.
29

---

E.3 Broader Impact and Potential Risk
CogVLA advances the efficiency and interpretability instruction-driven robotic manipulation. offering potential benefits in applications such as assistive robotics, household automation, and industrial assembly. Its biologically inspired sparsification and routing mechanisms reduce com- putation cost, making it more accessible for essource-constrainct platforms. However, as with any vision- iaiio-language-acttio system, risks include misinterpretation of ambiguous instructions, failure in unpredictable environments, and bias amplification from training data. If deployed in safety-critical settings without appropriate safeguards, such failures could lead to unintended behaviors or physical harm. We encourage the community to adopt robust evaluation protocols, prioritize transparency in model behavior, and consider human-in-the-lowe thoooopp designs to mitigate such risks. Broader societal considerations-inclludd data diversity, accessibility, and responsible deployment-shoul guide future development of systems built upon Cog VLA.
30


Reviews:
[{"review_id": "jF333jSSJj", "reviewer": "Reviewer_ER8U", "summary": "The authors present CogVLA, a novel Vision-Language-Action framework designed to address a critical bottleneck in the field: the immense computational cost and inefficiency of current state-of-the-art models. CogVLA introduces a cognition-aligned Vision-Language-Action framework that mirrors the human perceptual–reasoning–control loop through a three-stage, instruction-driven design. Stage 1 (Encoder-FiLM Aggregation Routing) embeds task instructions into the vision encoder to aggregate and compress visual tokens; Stage 2 (LLM-FiLM Pruning Routing) injects action intent into the language model to prune instruction-irrelevant tokens; and Stage 3 (V-L-A Coupled Attention) fuses the compressed multimodal context with bidirectional decoding for coherent, parallel action planning. By progressively pruning vision and language tokens and then fusing the compressed multimodal context, it cuts vision tokens 8×, FLOPs 3.1×, and training cost 2.5×, yet still achieves state-of-the-art success rates on LIBERO (97.4 %) and real-world ALOHA tasks (70 %). The work shows that instruction-aware sparsification can boost both efficiency and performance, advancing scalable embodied AI.", "questions": "None.", "limitations": "None.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros:\n1. The paper tackles one of the most pressing issues in embodied AI—the trade-off between model capability and computational feasibility. The proposed solution offers a path toward deploying powerful VLA models on resource-constrained platforms.\n2. The paper delivers on its promises. Achieving state-of-the-art performance while simultaneously delivering order-of-magnitude improvements in efficiency is a rare and significant contribution.\n\nCons:\n1. The model is tested on structured tasks from LIBERO and ALOHA. It is unclear how well the instruction-driven sparsification would generalize to tasks with fundamentally different structures or those requiring more implicit, common-sense reasoning not explicitly cued by the instruction. \n2. Following #1, the entire framework is predicated on the instruction being an accurate and sufficient guide. The system's performance would likely degrade sharply with ambiguous, or incomplete instructions. An analysis of this failure mode would strengthen the paper.\n3. The model introduces new, important hyperparameters, particularly the token retention ratio β in LFP-Routing and the sparsification ratio split between Stage 1 and 2 (Table 5). The paper demonstrates a good choice but does not provide a deeper analysis of the model's sensitivity to these parameters.", "was_revised": true, "final_justification": "Thanks to the authors for their detailed response to my concerns. The additional experiments provided effectively address the concerns I raised. As my previous rating 5 already adequately reflects the quality of the paper, I will keep my rating.", "rebuttal": "We thank your thoughtful feedback and for acknowledging our contribution toward balancing computational efficiency and task performance. Below, we address each concern in detail:\n\n### Cons 1: The model is tested ... instruction.\n\n### R1: We address this concern from two perspectives.\n\n> **(1) Generalization on structurally distinct tasks**:\n>\n> To evaluate CogVLA's generalization under underspecified and implicit instruction settings, we conducted additional experiments on **VLABench**, a recently introduced open-source benchmark for language-conditioned manipulation. Unlike prior benchmarks, VLABench features: \n>\n> - tasks requiring commonsense and world knowledge transfer,\n> - **natural, non-templated instructions with implicit human intent**, and \n> - joint evaluation of action policy and language understanding.\n>\n> For example, in the 'Select Book' suite, one task is: “**Take 'Pride and Prejudice' from the bookshelf.**” As shown below, CogVLA outperforms all baselines by a large margin:\n>\n> |Method|Select Book|Select Drink|Select Fruit|Select Tube|Add Condiment|Insert Flower|Overall|\n> |------|-----------|------------|------------|-----------|--------------|--------------|--------|\n> |Octo|0.0000|0.0000|0.0000|0.0154|0.0154|0.0000|0.0051|\n> |OpenVLA|0.0769|0.0846|0.0462|0.0769|0.1238|0.1385|0.0911|\n> |RDT-1B|0.0308|0.0769|0.1385|0.1238|0.2154|0.2154|0.1335|\n> |**CogVLA**|**0.40(+0.32)**|**0.68(+0.60)**|**0.62(+0.48)**|**0.36(+0.24)**|**0.78(+0.56)**|**0.61(+0.39)**|**0.61(+0.48)**|\n>\n> ------\n>\n> **(2) Real-world tests under abstract instructions**:\n>\n> In addition to the “T-shirt Folding” task (Table 2 of the main paper), which requires commonsense reasoning to interpret an abstract goal, we introduce two further ALOHA experiments involving implicit and underspecified instructions:\n>\n> - **“Clean up the table.”**: No objects or targets specified; requires collecting all clutter.\n> - **“Take the toy out of the drawer.”**: The toy is initially invisible; no explicit drawer-related actions (e.g., “open”) are mentioned.\n>\n> The results are below:\n>\n> ||Step 1|+Step 2|+Step 3|\n> |-|-------|--------|--------|\n> |Fold the T-shirt.|21/25|20/25|17/25|\n>\n> ||All Cleaned|1 Left↓|2 Left↓|≥3 Left↓|\n> |-|-----------|------|------|--------|\n> |Clean up the table.|20/25|2/25|1/25|2/25|\n>\n> | |Open Drawer|+Take Out Toy|\n> |-|-----------|-------------|\n> |Take the toy out of the drawer.|18/25|15/25|\n>\n> Note: The number of trials has been increased from 10 to 25 for stability. We will include visualizations of these new tasks in the revised submission.\n>\n\n------\n\n------\n\n### Cons 2: Following #1 ... paper.\n\n### R2: Analyzing robustness under ambiguous or incomplete instructions.\n\n> We conducted a failure analysis using the **VLABench** benchmark, which naturally features instructions with **implicit, metaphorical, or underspecified intent**. Below we present representative failure cases from three task suites, followed by successful examples where **CogVLA outperforms OpenVLA**, illustrating its improved reasoning capacity.\n>\n> **Failure Case Analysis of CogVLA:**\n>\n> 1. （Select Drink）instruction：“I'm craving some adrenaline for my upcoming gaming marathon.”\n>\n> 2. （Select Fruit ）instruction：“I want a fruit that symbolizes knowledge and education.”\n>\n> 3. （Select Fruit ）instruction：“The chicken wings are almost done; they just need a final touch to perfect their flavor profile.”\n>\n>     |ID|Ground Truth|Predicted|Failure Type|Analysis|\n>     |--|-------------|----------|-------------------------|-----------------------------------------------------------|\n>     |1|Monster|Juice|Metaphorical understanding|Metaphorical “adrenaline” poses a mild challenge; suggests improvement in **metaphor understanding and intent grounding**.|\n>     |2|Apple|Random|Symbolic reasoning|Symbolic “apple” shows gap in **analogy and cultural grounding**; room for enhancement.|\n>     |3|BBQ Sauce|Salt|Multi-target ambiguity|“Final touch” is vague; various condiments apply—**more context or user preference** would help disambiguate.|\n>\n> ------\n>\n> **Success Cases of CogVLA over OpenVLA:**\n>\n> 1. （Select Drink）instruction：“I want a refreshing beverage with natural fruit sugars. ”\n> 2. （Select Fruit ）instruction：“I need a fruit that is typically yellow when ripe.”\n> 3. （Add Condiment）instruction：“To enrich the flavors in this dish, we need something with a tomato base that's slightly sweet yet vinegary. Please add it.”\n>\n>     |ID|CogVLA Pred.|OpenVLA Pred.|Reasoning Type|Analysis|\n>     |--|--------------|----------------|------------------------|-----------------------------------------------------------|\n>     |1|Juice|Cola|Functional semantics|**CAtten enhances grounding; “fruit sugars” better aligns with juice than soda.**|\n>     |2|Banana|Pear|Commonsense reasoning|**EFA-Routing preserves yellow-related tokens, guiding correct banana grounding.**|\n>     |3|Ketchup|Salt|Multi-attribute reasoning|**CogVLA integrates cues (“tomato,” “sweet,” “vinegary”) → ketchup; shows strong compositional reasoning.**|\n>     \n>     Note: The OpenVLA baseline is from the official VLABench repository.\n>\n> We will include frame-wise heatmaps and pruning-confidence curves for the above cases in the revised appendix, along with additional examples across task suites. These failure modes also reveal future research directions: improving metaphor and event-grounded reasoning, integrating symbolic knowledge, and modeling user preference under ambiguous intent.\n>\n\n------\n\n------\n\n### Cons 3: The model introduces ... parameters.\n\n### R3: Clarifications on hyperparameter design are as follows.\n\n> **(1) Hyperparameter β in LFP-Routing**:\n>\n> We use a shifted cosine schedule (details in **Appendix A.1**) to modulate token retention across layers, with β constrained to [0.05, 0.85] to avoid degenerate pruning. We conducted two experiments varying β, and the results exhibit high stability:\n>\n> |Clamp Range|Approx.Pruning Rate|Spatial SR|\n> |-----------|-------------------|-----------|\n> |[0.05,0.85]|0.533|98.6|\n> |[0.10,0.80]|0.536|98.6|\n> |[0.15,0.75]|0.539|98.4|\n>\n> ------\n>\n> **(2) Sparsification ratio between Stage 1 and Stage 2**:\n>\n> To achieve a target 8× sparsification (12.5% retention), we fix Stage 2 at 0.533 (in line with the β experiments above) and vary Stage 1 rates. The results show consistent performance:\n>\n> |Stage1 Aggregation|Stage2 Pruning|Spatial SR|\n> |------------------|---------------|-----------|\n> |0.250|0.533|98.6|\n> |0.234|0.533|98.4|\n> |0.266|0.533|98.6|\n>\n> Note: Total retention ≈ 0.125. Stage 2 fixed; Stage 1 varied.\n>\n> ------\n>\n> **(3) Ablation Study of Hyperparameters in Appendix C.3**:\n>\n> These results validate that the chosen hyperparameters enable substantial sparsity while preserving task performance. \n>\n> |Stage1|Stage2|Total Sparsification|Spatial SR|\n> |------|------|--------------------|-----------|\n> |2×|2×|4×|96.2|\n> |4×|4×|16×|93.2|\n> |2×|4×|8×|94.6|\n> |**4×**|**2×**|**8×**|**98.6**|\n>\n\n\nWe sincerely thank you again for the constructive feedback, which has significantly contributed to improving our work. We remain open to further suggestions and will actively incorporate them in future revisions."}, {"review_id": "IFBQ3u1Q7l", "reviewer": "Reviewer_iG5e", "summary": "This paper introduces CogVLA, a vision-language-action (VLA) model that uses sparsification to improve efficiency and performance. They propose an Encoder-FiLM based aggregation rounting (EFA-Routing), a LLM-FiLM based pruning (LFP-Routing) and a VLA coupled attention (CAtten) to achieve this. They conduct experiments on LIBERO benchmark and real-world tasks, showing that CogVLA outperforms existing VLA models in terms of efficiency and performance.", "questions": "Answer all the questions in Weaknesses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The paper is well-written and structured, making it easy to follow the proposed approach.\n2. The authors provide a clear motivation for the use of sparsification in VLA models.\n\nWeaknesses\n1. The methodology novelty is limited. \n1) The CAtten seems already proposed in $\\pi_0$. $\\pi_0$ proposes to use bi-directional attention within each modality, i.e., the action decoding is actually parellel and requires only one feed-forward pass. The authors should clarify how their CAtten differs from $\\pi_0$.\n2) The proposed EFA-Routing and LFP-Routing focus on the same sparsification problem as in VLM models, which has been extensively studied in the literature. VLMs also study fine-grained instruction-driven reasoning, so the challenges faced by both VLA and VLM models are similar. The authors should clarify how their approach differs from existing methods in VLMs and why it is suitable for VLA models. There seems to be no specilized design for VLA models. The authors should clarify how their approach differs from existing methods in VLMs and why it is suitable for VLA models.\n2. The experiments are not convincing.\n1) The authors choose OpenVLA-style as the main baseline for efficiency evaluation, which is not a strong baseline. $\\pi_0$ removes auto-regressive decoding and uses flow-matching to predict continuous action, which is effective and much more efficient than OpenVLA-style (70ms vs 250ms). The authors should include $\\pi_0$ as a baseline to demonstrate the efficiency and effectiveness of their method.\n2) The evaluation is conducted on small number of trials (10 for each task in the real world), which may not be sufficient to draw reliable conclusions. On LIBERO benchmark, the proposed method achieves 0.3% improvement over OpenVLA-OFT, which raises questions about whether this is due to randomness or the effectiveness of the proposed method. The authors should report the standard deviation of the results and conduct more trials to ensure the reliability of the results.", "was_revised": true, "final_justification": "The rebuttal solves my concern about its improvement over the state-of-the-art VLA models on both efficiency and performance, so I raise my rating to 4.", "rebuttal": "We sincerely thank your constructive feedback. We appreciate the recognition of our writing clarity and the motivation behind introducing sparsification in VLA models. Below, we address each concern in detail.\n\n###  W 1.1: The CAtten seems ... from $\\pi_0$.\n\n### R 1.1:  Two key distinctions are outlined below.\n\n> **(1) Attention Design and Semantic Consistency**\n>\n> Please refer to **Figure 3 (CAtten vs. other methods)** in the paper, which illustrates two key features of CAtten: i) a hierarchically structured attention mask, and ii) the injection of action intent from instruction into observation tokens via LLM-FiLM modulation. Below, we further provide a comparison with \\$\\pi\\_0\\$, and will include its attention visualization in the **updated Figure 3**.\n>\n> - $\\pi_0$ adopts **bidirectional attention** within three independent blocks: `[Observation, Instruction]`, `[Robot State]`, and `[Action]`. In contrast, **CAtten employs unidirectional causal attention** within `[Observation, Instruction]` and `[Robot State]`, maintaining compatibility with pretrained LLMs and supporting temporally coherent reasoning.\n> - $\\pi_0$ treats `[Observation, Instruction]` as a single attention block with bidirectional attention, which disrupts the directional flow of semantic reasoning. In contrast, **CAtten separates `[Observation]` and `[Instruction]` and injects action intent from `[Instruction]` into `[Observation]` via LLM-FiLM**, thereby avoiding this issue and enhancing cross-modal consistency.\n>\n> ------\n>\n> **(2) Action Decoding Architecture**\n>\n> - $\\pi_0$ adopts a dual-system Mixture-of-Experts (MoE) design: a shared VLM backbone and an independent action expert. While action decoding leverages flow matching, it demands **1 forward pass through the VLM** and **10 through the action expert**.\n> - In contrast, CogVLA adopts a unified architecture where `[Observation, Instruction], [Robot State] and [Action]`are processed within **a single LLM context window, enhancing reasoning capability**. Actions are decoded as continuous outputs via L1 regression over action chunks, requiring only **a single forward pass**.\n> - Notably, $\\pi_0$’s action expert operates outside the VLM context, constraining its generalization to diverse or long-horizon tasks. This limitation is reflected on the LIBERO-Long benchmark, where CogVLA delivers superior performance: \n>\n>     |             | $\\pi_0$          | $\\pi_0$+FAST     | CogVLA   |\n>     | ----------- | ---------------- | ---------------- | -------- |\n>     | LIBERO-Long | 85.2 **(−10.2)** | 60.2 **(−35.4)** | **95.4** |\n\n------\n\n------\n\n### W 1.2: The proposed EFA-Routing and LFP-Routing ... VLA models.\n\n### R 1.2: Three key innovations distinguish our design from prior sparsification methods in VLMs.\n\n> ||VLMs|CogVLA|Suitability for VLA|\n> |--|-|-|-|\n> |Decoding Paradigm|Autoregressive|Parallel decoding|**Reduces inference latency for real-time robot control**|\n> |Input-side Sparsification|Focus on instruction-relevant regions|**EFA-Routing**: Instruction-driven (1) Local anchoring; (2) Global action cues|**Aligns token selection with manipulation semantics**|\n> |In-LLM Sparsification|Tokens unaware of instructions; text-supervised pruning|**LFP-Routing**: (1) Intent via LLM-FiLM; (2) Action-pruned|**Tri-modal alignment for instruction-conditioned action**|\n> |Progressive Sparsification|Absent|**EFA-Routing→LFP-Routing** (bio-inspired)|**Reflects cognitive stages in manipulation (VAS→SMA)**|\n>\n> We elaborate on the design contributions below:\n>\n> **(1) Input-side Task-guided Sparsification (EFA-Routing)**:\n>\n> EFA-Routing explicitly establishes a strong Observation-Instruction-Action linkage in VLA tasks by extracting:\n>\n> - **Local object anchoring** via DINOv2 path;\n> - **Global action cues** via SigLIP path;\n> - These complementary features are adaptively fused through an Aggregation Router, which **balances their contributions**. \n>\n> Visualizations of heatmaps (**Appendix D.2**) demonstrate that the retained tokens are both manipulation task-relevant and spatially discriminative.\n>\n> ------\n>\n> **(2) In-LLM Instruction-driven Sparsification (LFP-Routing)**:\n>\n> LFP-Routing performs structured token pruning within the LLM, conditioned on instruction semantics and action supervision. Through **LLM-FiLM modulation**, it injects goal intent into the shared context, forming a unified **instruction-observation-action tri-modal** reasoning mechanism tailored for VLA, which is absent in conventional VLM approaches.\n>\n> ------\n>\n> **(3) Progressive and Biologically Motivated Sparsification**:\n>\n> - EFA-Routing and LFP-Routing operate in a progressive fashion (**EFA-Routing → LFP-Routing**), reflecting cognitive-stage alignment (**VAS → SMA**). \n> - This cognitive alignment enhances efficiency without compromising action consistency, outperforming other efficiency-oriented VLA models such as DeeR-VLA, MoLe-VLA, and RoboMamba on both computational and functional metrics.\n> - CogVLA demonstrates a more **human action-like design** philosophy aligned with the intrinsic characteristics of robotic manipulation.\n>\n\n------\n\n------\n\n### W 2.1: The authors choose OpenVLA-style ... their method.\n\n### R 2.1: We position CogVLA as a cognitively aligned evolution of single-system VLA architectures.\n\n> In end-to-end VLA models, OpenVLA-style and $\\pi_0$-style architectures represent two dominant paradigms, where OpenVLA excels in reasoning and $\\pi_0$ offers higher inference efficiency.\n>\n> **(1) We include $\\pi_0$ as a baseline for efficiency comparison as shown below.** \n>\n> |Method|LLM|Train Steps↓|Training Time (per 10k)↓|Relative Training Cost↓|Latency↓|\n> |-|-|-|-|-|-|\n> |$\\pi_0$|2.6B|800–1200k|4.3 h|~14.1×|**0.078 s**|\n> |$\\pi_0$+FAST|2.6B|80–200k|6.3 h|~2.9×|0.329 s|\n> |OpenVLA|7B|-|-|-|0.254 s|\n> |OpenVLA-OFT|7B|50–150k|12.5 h|~4.0×|0.132 s|\n> |**CogVLA**|**7B**|**50–80k**|**4.7 h**|**1 (baseline)**|**0.091 s**|\n>\n> - For **training efficiency**, compared to OpenVLA-OFT and CogVLA, $\\pi_0$ requires an order of magnitude more steps.\n> - For **inference efficiency**, while CogVLA is slightly slower than $\\pi_0$, it uses a larger 7B backbone (vs. $\\pi_0$’s 2.6B), yet still achieves the best overall trade-off between performance and efficiency.\n> - As $\\pi_{0.5}$ has not been open-sourced, we do not include efficiency comparisons with it.\n>\n> **(2) We include $\\pi_0$ and $\\pi_{0.5}$ as baselines for performance comparison on LIBERO benchmark.** Despite partial decoupling or gradient blocking, these variants still suffer from weakened semantic consistency and reduced reasoning capacity, especially on long-horizon tasks (LIBERO-Long):\n>\n> |Method|LIBERO-Spatial|LIBERO-Object|LIBERO-Goal|LIBERO-Long|LIBERO-Overall|\n> |-|-|-|-|--|-|\n> |$\\pi_0$|96.8|**98.8**|95.8|85.2 **(−10.2)**|94.2|\n> |$\\pi_0$+FAST|96.4|96.8|88.6|60.2 **(−35.4)**|85.5|\n> |$\\pi_{0.5}$(scratch)|96.6|97.2|94.6|84.8 **(−10.4)**|92.7|\n> |$\\pi_{0.5}$(gen. model)|98.0|97.8|95.6|85.8 **(−9.4)**|96.0|\n> |**CogVLA**|**98.6**|**98.8**|**96.6**|**95.4**|**97.4**|\n>\n\n------\n\n------\n\n### W 2.2: The evaluation is conducted ... the results.\n\n### R 2.2: We provide additional experimental validation to address this concern.\n\n> **(1) Extended Real-World Trials** \n>\n> - We have expanded the real-world evaluations **from 10 to 25 trials** under the same training setup. Results are shown below:\n>\n>     |Method|Cube→Plate+Toy→Bowl|Open+Place+Close|Step1+Step2+Step3|Avg SR|\n>     |--|--|-|--|-|\n>     |OpenVLA-OFT|19/25+17/25|20/25+16/25+13/25|18/25+15/25+11/25|54.7%|\n>     |CogVLA|23/25+22/25|21/25+19/25+17/25|23/25+21/25+17/25|**74.7%**|\n>\n> ------\n>\n> **(2) Statistical Reliability** \n>\n> - We report standard deviation across three seeds on the LIBERO benchmark in **Appendix C.1** to ensure statistical reliability:\n>\n>     ||Spatial|Object|Goal|Long|Average|\n>     |--|--|-|--|-|-|\n>     |**CogVLA**|98.5±0.5|98.8±0.4|96.5±0.6|95.2±1.1|**97.4±0.4**|\n>     \n>     The low standard deviation (±0.4) confirms that CogVLA's performance advantage is consistent and not attributable to random variation.\n>\n> ------\n>\n> **(3) Broader Benchmarking on VLABench** \n>\n> - VLABench is a recently benchmark designed to test **world knowledge and common-sense reasoning**. We compare CogVLA with the official baselines reported in the VLABench paper. Across all subtasks, CogVLA delivers dramatic improvements over baselines.\n>\n>    |Method|Select Book|Select Drink|Select Fruit|Select Tube|Add Condiment|Insert Flower|Overall|\n>    |-|--|-|-|-|-|-|-|\n>     |Octo|0.0000|0.0000|0.0000|0.0154|0.0154|0.0000|0.0051|\n>     |OpenVLA|0.0769|0.0846|0.0462|0.0769|0.1238|0.1385|0.0911|\n>     |RDT-1B|0.0308|0.0769|0.1385|0.1238|0.2154|0.2154|0.1335|\n>     |**CogVLA**|**0.40(+0.32)**|**0.68(+0.60)**|**0.62(+0.48)**|**0.36(+0.24)**|**0.78(+0.56)**|**0.61(+0.39)**|**0.61(+0.48)**|\n\nWe hope these clarifications adequately address your concerns. And we remain open to further suggestions and will actively incorporate them in future revisions."}, {"review_id": "70jg2PxqTJ", "reviewer": "Reviewer_Bu5w", "summary": "This paper proposes a novel instruction-driven VLA framework, CogVLA, which achieves efficient cross-modal sparsification and semantic alignment through a three-stage progressive design (EFA-Routing, LFP-Routing, CAtten). The framework adopts a biologically inspired architecture that simulates human multimodal coordination mechanisms to enhance the coherence of perception-reasoning-control. The method achieves SOTA performance on the LIBERO benchmark and real-robot tasks, while significantly reducing training costs and inference latency.", "questions": "1. I hope the authors will carefully consider the first **Major Weakness**. While the paper itself cannot be changed during review, I would be willing to increase my score if the rebuttal addresses this concern thoughtfully.\n\n2. I would like the authors to supplement their results with evaluations on additional VLA datasets and include GPU-wise comparisons for the reported efficiency gains.\n\n3. I would appreciate it if the authors could specify how they plan to improve the clarity of figures and tables in future versions of the paper.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n- **Significant gains in both efficiency and performance**:  \n  The method achieves a success rate of 97.4% on LIBERO and 70.0% on real-robot tasks. Compared to OpenVLA, it reduces training cost by 2.49× and inference latency by 2.79×, making it a rare case of simultaneous improvement in both speed and effectiveness.\n\n- **Original method design**:  \n  The three-stage architecture and cognitive alignment modules are innovative and well-supported by extensive ablation studies.\n\n- **Clear writing and detailed supplementary materials**:  \n  The paper is well-structured and accessible, which facilitates reader understanding.\n\n---\n\n## Weaknesses\n\n### Major Weaknesses\n\n1. **Insufficient integration of biological motivation and experimental validation**:  \n   The three-stage cognitive alignment design of CogVLA is framed using biological terminology (VAS, SMA, PMC, etc.) inspired by human cognitive processes. However, the paper lacks either of the following:\n\n   - **Solid support from cognitive science theory**:  \n     While the paper cites relevant literature (e.g., references [24], [43]), it does not clearly explain how these biological concepts are incorporated into the VLA design. As a result, the biological analogy feels more like a conceptual metaphor than a theoretically grounded design principle.\n\n   - **Experimental validation of the biological motivation**:  \n     A compelling way to justify such biologically inspired design would be through empirical results—e.g., showing that CogVLA achieves performance that mirrors certain aspects of human behavior, or that its design leads to improvements specifically in tasks known to benefit from human-like reasoning. Unfortunately, the current version of the paper does not include such comparative analyses, either quantitatively or qualitatively (e.g., visualizations).\n\n2. **Limited comparative experiments**:  \n   The evaluation in simulation is limited to the LIBERO dataset. Although LIBERO is widely used, the performance gap between CogVLA and OpenVLA-OFT is relatively small. Evaluating on additional VLA benchmarks (e.g., BridgeData V2, CALVIN) would better demonstrate the generality and robustness of CogVLA.\nIn Table 4, the results seem reveal that the stage 3 is critical with improvement of 6.6%. It is hard to understand why the CAtten can bring such considerable improvements. Also the other results indicate that each component is fluctuant.\n\n\n3. **Lack of hardware variation in efficiency results**:  \n   The **Efficiency Optimization Results** section lacks comparative analysis across different GPUs. Since reduced training cost and inference latency are claimed as major contributions, hardware configuration becomes a critical factor and should be considered explicitly. This paper also mentions the other speed up methods, such as MoD. However, it does not compare with these methods.\n\n\n---\n\n### Minor Weaknesses\n\n- **Generalization remains underexplored**:  \n  As acknowledged by the authors, “its performance under out-of-distribution instructions or unseen manipulation categories is yet to be thoroughly evaluated.” Including such results would further support the biologically inspired design's validity, especially for tasks humans perform naturally but current SOTA methods often fail.\n\n- **Missing explanations in figures**:  \n  Some key diagrams (e.g., Figures 2 and 3) lack detailed annotations for arrows and symbols, which hinders reader comprehension. Figure 1 is too compact to see the details clearly. In section 2, the “I”and “t”in eqation 1 are not be defined.\n\n- **Typographical error**:  \n  In Table 3, “Taining Cost” should be corrected to “Training Cost”.", "was_revised": true, "final_justification": "I raise my score to 4： Borderline accept. Some limites and errors should be refined in future  versions.", "rebuttal": "Thank you for the detailed and constructive feedback. We appreciate your recognition of our contributions and welcome the valuable suggestions for improvement.\n\n### W1: Insufficient integration... validation.\n\n### R1: We address this concern through both theoretical grounding and experimental validation.\n\n**(1) Theoretical Foundation: Cognitive Science-Inspired Architecture**\n\n|Index|Human→Robot|Stage|\n|-|-|-|\n|(1.1)|VAS→EFA-Routing|Perception|\n|(1.2)|SMA→LFP-Routing|Intention|\n|(1.3)|PMC→CAtten|Reasoning|\n\nWe will add a figure illustrating the mapping from human cognition **(VAS → SMA → PMC)** to CogVLA modules **(EFA-Routing → LFP-Routing → CAtten)** in the main paper.\n\n> **(1.1) VAS → EFA-Routing**\n>\n>\n> In the human brain, VAS selectively enhances perception of task-relevant features through top-down signals from the prefrontal and parietal cortex, while suppressing distractors. This mechanism improves semantic alignment and efficiency in visuomotor execution, especially under complex scenes. **To emulate this, EFA-Routing:**\n>\n> - Uses Encoder-FiLM to modulate visual encoding based on **instruction semantics**.\n> - Aggregates 25% of patch tokens via autoregressive attention, **reducing redundancy**.\n> - Combines local (DINOv2) and global (SigLIP) features via an Aggregation Router, enabling **spatial precision and semantic grounding**.\n>\n> **We visualize its token selection via heatmaps in Appendix D.2, demonstrating its effectiveness in implementing VAS-like behavior.**\n>\n> ------\n>\n> **(1.2) SMA→LFP-Routing**\n>\n> SMA is critical for multi-step planning and intention modeling. It integrates multimodal context and high-level goals, often activating even before actual action, leveraging prior knowledge to form action intentions before execution. **To emulate such prior intent construction, LFP-Routing:**\n>\n> - Uses LLM-FiLM to **inject instruction-driven intent** into observation tokens within the shared LLM context without disrupting LLM reasoning (avoiding bidirectional attention across modalities).\n> - Prunes 50% of observation tokens via action supervision, **improving alignment and efficiency** while preserving reasoning fidelity.\n>\n> ------\n>\n> **(1.3) PMC→CAtten**\n>\n> PMC generates executable, fluent motor plans based on current observations. It supports high-level reasoning during transitions (e.g., from “pick” to “place”), while allowing low-level motion execution to proceed smoothly. **CAtten mimics PMC by combining two complementary attention mechanisms:**\n>\n> - Causal Vision-Language Attention for reasoning and intent alignment;\n> - Bidirectional Action Chunk Attention for fluent and efficient decoding.\n>\n> These are integrated in a Unified Hybrid Attention within a shared context window, achieving a balance between reasoning and real-time execution.\n>\n\nCogVLA’s **EFA-Routing**, **LFP-Routing**, and **V-L-A Coupled Attention** reflect the roles of VAS, SMA, and PMC in human cognition. We believe modeling human cognitive pathways offers a promising direction for embodied intelligence. \n\n**(2) Empirical Validation of Cognitive Design**\n\n> **(2.1) Quantitative Evaluation on Cognition-Level Tasks**\n>\n> VLABench is designed to evaluate **world knowledge and commonsense reasoning** through implicit, non-templated instructions, making it ideal for testing cognition-level capabilities. Compared with official baselines, **CogVLA achieves significant absolute gains across all tasks**:\n>\n> |Method|Select Book|Select Drink|Select Fruit|Select Tube|Add Condiment|Insert Flower|Overall|\n> |-|-|-|-|-|-|-|-|\n> |Octo|0.0000|0.0000|0.0000|0.0154|0.0154|0.0000|0.0051|\n> |OpenVLA|0.0769|0.0846|0.0462|0.0769|0.1238|0.1385|0.0911|\n> |RDT-1B|0.0308|0.0769|0.1385|0.1238|0.2154|0.2154|0.1335|\n> |**CogVLA**|**0.40(+0.32)**|**0.68(+0.60)**|**0.62(+0.48)**|**0.36(+0.24)**|**0.78(+0.56)**|**0.61(+0.39)**|**0.61(+0.48)**|\n>\n> Note: **For instance, the instruction \"A burger on the table, but it feels like something's missing\" from the \"Select Drink\" task implicitly requires the robot to fetch a Cola.**\n>\n> ------\n>\n> **(2.2) Quantitative Evaluation on Reasoning-Intensive Tasks**\n>\n> The LIBERO-Long benchmark emphasizes reasoning with extended temporal dependencies, which are central to human cognition. We evaluate CogVLA against over 50 state-of-the-art models (top 10 shown below; full results in the revised submission). CogVLA outperforms all existing methods.\n>\n> |Model|CogVLA|OpenVLA-OFT|UniVLA|PD-VLA|VOTE|UVA|LBP|Seer|OpenVLA-FreqPolicy|BEAST-F|\n> |-|-|-|-|-|-|-|-|-|-|-|\n> |SR(%)|95.4|94.5|92.0|91.7|91.0|90.0|88.6|87.7|87.6|86.4|\n>\n> ------\n>\n> **(2.3) Qualitative Analysis of Human-Like Visual Alignment**\n>\n> - In **Appendix D.2**, we visualize **attention heatmaps** produced by CogVLA to assess whether the perception module focuses on **instruction-relevant regions**. \n> - Additional visualizations from CALVIN, VLABench, and real-world settings will be included in the revised submission.\n>\n> ------\n>\n> **(2.4) Qualitative Analysis of Human-Like Error Correction Behavior**\n>\n> - In both simulation and real-world deployment, **CogVLA consistently retries failed actions**, despite no explicit supervision. In contrast, other models proceed without correction. We attribute this emergent behavior to CogVLA’s **structured intent modeling and observation-action grounding**, enabling reflective reassessment of action correctness.\n> - Such **self-initiated error detection and rollback** is rare in models without cognitively structured planning. **We will release video demonstrations on our project website.**\n>\n> ------\n>\n> **(2.5) Ablation of EFA-Routing → LFP-Routing → CAtten**\n>\n> **Table 4 of the paper** presents ablations validating the effectiveness of each module in our cognitively inspired architecture.\n\n------\n\n------\n\n### W2: Limited comparative experiments.\n\n### R2: We address this with a broader range of evaluations.\n\n**(1) Broader Benchmarking**\n\n> **(1.1) CALVIN Benchmark (ABC→D) Evaluation.** \n>\n> CogVLA demonstrates competitive performance. The following results are adapted from the UniVLA paper (**RSS 2025, May 2025**, Learning to Act Anywhere with Task-centric Latent Actions):\n>\n> |Method|1|2|3|4|5|Avg.Len.|\n> |-|-|-|-|-|-|-|\n> |RT-1|53.3|22.2|9.4|3.8|1.3|0.90|\n> |RoboFlamingo|82.4|61.9|46.6|33.1|23.5|2.48|\n> |SuSIE|87.0|69.0|49.0|38.0|26.0|2.69|\n> |GR-1|85.4|71.2|59.6|49.7|40.1|3.06|\n> |OpenVLA|91.3|77.8|62.0|52.1|43.5|3.27|\n> |CLOVER|96.0|83.5|70.8|57.5|45.4|3.53|\n> |RoboDual|94.4|82.7|72.1|62.4|54.4|3.66|\n> |UniVLA|95.5|85.8|75.4|66.9|**56.5**|3.80|\n> |**CogVLA**|**97.2**|**86.8**|**76.0**|**67.2**|56.2|**3.83**|\n>\n> Note: While CALVIN favors large-scale pretraining, CogVLA achieves state-of-the-art results by fine-tuning on ABC and testing on D **without any pretraining**.\n>\n> ------\n>\n> **(1.2) VLABench Evaluation.**\n>\n> Please refer to **(R1-2.1)**.\n>\n> ------\n>\n> **(1.3) Real-world Evaluation.**\n>\n> We further extended real-world experiments on the ALOHA platform **from 10 to 25 trials**, demonstrating the stability and consistency of CogVLA's performance.\n>\n> ||Cube→Plate+Toy→Bowl|Open+Place+Close|Step1+Step2+Step3|Avg SR|\n> |-|-|-|-|-|\n> |CogVLA|23/25+22/25|21/25+19/25+17/25|23/25+21/25+17/25|**74.7%**|\n>\n\n**(2) Consistency Analysis of Fluctuant**\n\n> **(2.1) Consistency Across Seeds**\n>\n> **Table 1 in Appendix shows CogVLA consistently low variance across LIBERO task subsets**, with an average success rate of 97.4% (mean ± std over 3 seeds), indicating strong generalization and reliability.\n>\n> |Method|Spatial|Object|Goal|Long|Average|\n> |-|-|-|-|-|-|\n> |CogVLA|98.5±0.5|98.8±0.4|96.5±0.6|95.2±1.1|**97.4±0.4**|\n>\n> ------\n>\n> **(2.2) Cascaded Stability**\n>\n> The CAtten module introduces a coupled attention mechanism that preserves fine-grained cross-modal alignment and coherent action planning, even with heavily pruned visual input—a capability beyond standard attention. **Its role is essential within the EFA-Routing → LFP-Routing → CAtten cascade and cannot be replaced by earlier stages.**\n\n------\n\n------\n\n### W3: Lack of hardware variation in efficiency results.\n\n### R3: Two main points as follows.\n\n> **(1) Efficiency Across Multiple GPU Configurations**\n>\n> As shown below, CogVLA achieves consistently superior performance across hardware variants:\n>\n> |Method|GPU|Train Cost↓|Infer Time↓|Throughput↑|\n> |-|-|-|-|-|\n> |OpenVLA-OFT|L40s|10.8h/10k|0.127s|63.0Hz|\n> |**CogVLA**|**L40s**|**7.2h/10k**|**0.088s**|**90.9Hz**|\n> |OpenVLA-OFT|A800|12.5h/10k|0.132s|60.6Hz|\n> |**CogVLA**|**A800**|**4.7h/10k**|**0.091s**|**87.9Hz**|\n>\n> Note: L40s results use reduced batch size. CogVLA benefits more from larger batches due to sparsity.\n>\n> ------\n>\n> **(2) Comparison with Other Acceleration Methods (LIBERO-Spatial)**\n>\n> **We report comparisons with two other acceleration methods in Table 6 of the paper.** Here, we further include MoD, where CogVLA demonstrates a clear advantage.\n>\n> |Method|CogVLA|FastV|SliME|MoD|\n> |-|-|-|-|-|\n> |SR(%)|98.6|88.2|77.6|81.0|\n>\n\n------\n\n------\n\n### Minor W1: Generalization remains underexplored\n\n> **R1:** We added results on out-of-distribution instructions and unseen categories. See **W2-(1)** for the ABC→D evaluation on the CALVIN benchmark, where CogVLA shows strong zero-shot generalization.\n\n### Minor W2: Missing explanations in figures\n\n> **R2:** Thank you for pointing this out. We’ve added missing annotations, revised Figure 1 and Equation (1), and improved overall clarity. A full revision will be submitted.\n\n### Minor W3: Typographical error\n\n> **R3:** The typo has been corrected and all tables rechecked. We appreciate your careful feedback.\n\n------\n\n------\n\nWe sincerely thank you for the insightful suggestions, which have been invaluable in refining the paper’s structure and empirical support. We hope our responses sufficiently address your concerns."}, {"review_id": "iS57mq0O9Y", "reviewer": "Reviewer_EsjF", "summary": "The paper introduce CogVLA, a Cognition-Aligned Vision-Language-Action framework that leverages instruction-driven routing and sparsification to improve both efficiency and performance. Specifically, it contains of three modules and innovations: 1) Encoder-FiLM based Aggregation Routing, 2)LLM-FiLM based Pruning Routing and V‑L‑A Coupled Attention.", "questions": "See Above.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This paper is clearly written. \n2. It proposes a combination approach by incorporation of Encoder-FiLM based Aggregation Routing, LLM-FiLM based Pruning Routing and a novel V‑L‑A Coupled Attention for VLA framework. \n3. Experimental results on LIBERO and real-world tasks demonstrate its effectiveness, while also being efficient. \n\nWeakness: \n1. Despite noted as Cognition-Aligned VLA model, I find it hard to synergy author's three approachs into a coherent CogVLA framework. \n2. The method is more combinative and separated contributions instead of totally novel. For example, the use aggregating different visual features is an intuitive method by various approach needs multiple level feature map such as Depth Transformer. The attention mask approach is also very straightforward. \n3. How do this model compare with large scale trained models such as Pi series and Gemini Robotics? The author may note the scale of training compute differences and capability differences.", "was_revised": true, "final_justification": "After reviewing the author's rebuttal, I decide to maintain my score which still holds an up-word rating about this paper with borderline accept.", "rebuttal": "We thank you for your positive recognition of the paper's structure, clarity, and methodological effectiveness, as well as the thoughtful and critical insights on CogVLA. Below is our detailed response to the identified concern:\n\n### W1: Despite noted as ... framework.\n\n### R1: Clarifying the cognition-aligned architecture through perception–intention–reasoning mapping.\n\n> The architecture of CogVLA is not a simple combination of three techniques. Rather, it is systematically constructed based on the division of labor in human multimodal cognition—specifically the perception–intention–reasoning pathway. **The three modules in CogVLA have clearly defined functional correspondences and semantic dependencies**, as shown below:\n>\n> | Human Cognitive Pathway| CogVLA Module   | Functional Stage       |\n> | ------------------------------ | --------------- | ---------------------- |\n> | VAS (Visual Attention System)  | **EFA-Routing** | **Perceptual Aggregation** |\n> | SMA (Supplementary Motor Area) | **LFP-Routing** | **Intention Construction** |\n> | PMC (Premotor Cortex)          | **CAtten**      | **Reasoning & Execution**  |\n>\n> We will include this mapping diagram in the revised version to clarify the cooperative structure among modules. Detailed explanations are as follows:\n>\n> - **Module Synergy**:\n>      **EFA-Routing emulates the VAS** by aggregating task-relevant visual features through instruction-guided modulation; **LFP-Routing corresponds to the SMA**, injecting linguistic intent into perceptual representations and pruning redundant tokens to form executable motor plans; **CAtten mirrors the PMC**, integrating linguistic reasoning and action decoding within a shared context to ensure coherent and semantically aligned action generation. **These three components collaboratively form a brain-inspired perception–intention–execution pipeline.**\n> - **Information Flow**:\n>      **CogVLA adopts a strictly cascaded structure**: tokens selected by EFA-Routing serve as the sole input to LFP-Routing, and the pruned outputs from LFP-Routing are the exclusive decoding source for CAtten. This ensures a unified and consistent architecture, rather than a plug-and-play combination of independent methods.\n>\n> In essence, the **EFA-Routing → LFP-Routing → CAtten** flow in CogVLA corresponds directly to the human **VAS → SMA → PMC** pathway. This end-to-end cognitive alignment is the defining characteristic of our Cognition-Aligned VLA framework.\n>\n\n------\n\n------\n\n### W2: The method is more ... straightforward.\n\n### R2: Explaining the integrative innovations behind aggregation, pruning, and attention mechanisms.\n\n> As discussed in R1, CogVLA is not a naïve combination of existing techniques. Its advantage arises from **a cognitively aligned, three-stage collaborative design**. While we acknowledge that some sub-functionalities build upon established prior work, our specific module configurations, architectural composition, and control logic are novel. We elaborate on the following key innovations:\n>\n> **(1) Aggregation of Visual Features**\n>\n> - **Motivational Innovation**: Instruction-aligned token aggregation.\n>      Our goal in visual feature aggregation is not merely to sparsify the token set, but to retain tokens that are semantically aligned with task instructions. **As visualized in Appendix D.2, the retained 1/8 of visual tokens preserve both local object anchors and global action-relevant cues, demonstrating semantically meaningful sparsification.**\n> - **Design Innovation**: Progressive, intra- and inter-encoder aggregation.\n>      EFA-Routing conducts visual token aggregation **in two stages**—first within a single visual encoder, then across two different encoders. **The intra-encoder aggregation follows an autoregressive structure, while inter-encoder fusion is guided by adaptive weighting, enabling complementary feature integration across distinct encoding branches.**\n>\n> **(2) Attention Mask Strategy**\n>\n> - **Motivational and Design Innovation**: Joint improvement of reasoning ability and inference efficiency.\n>      In VLA tasks, attention windows span `[observation-instruction-action]`, where action tokens possess unique temporal and semantic roles. Our CAtten module adopts a tailored attention design:\n>     - Causal autoregression within `[observation]` and `[instruction]` preserves the reasoning capacity inherited from pretrained LLMs.\n>     - Bidirectional attention within `[action]` allows parallel decoding, enhancing sequence continuity and inference efficiency.\n>     - Task intent is injected from `[instruction]` to `[observation]` via LLM-FiLM, avoiding the performance degradation seen in $\\pi_0$ and OpenVLA-OFT caused by full bidirectional attention across `[observation, instruction]`.\n>\n> **(3) Unified Cognition-Aligned Framework**\n>\n> - We believe CogVLA’s core contribution lies in realizing a cognition-aligned architecture that is not only theoretically grounded through explicit cognitive mappings, but also empirically validated. It achieves **a triple advantage in performance, efficiency, and generalization**—which would be difficult to attain through a simple additive combination of components.\n>\n> The idea of modeling robotic manipulation based on human cognitive and motor processes is both promising and impactful, opening avenues for deeper understanding of embodied intelligence.\n\n------\n\n------\n\n### W3: How do this model ... differences.\n\n### R3: Comparing training efficiency and long-horizon performance with $\\pi$ series and Gemini Robotics.\n\n> We thank the reviewer for raising this important yet previously under-discussed dimension. Below, we compare CogVLA with the $\\pi$ series and Gemini Robotics in terms of both training compute and model capability.\n>\n> **(1) $\\pi$ Series**\n>\n> - **Training Compute**:\n>\n>     Compared to the $\\pi$ series, CogVLA employs a larger LLM backbone but still achieves superior training efficiency, significantly reducing GPU time and computational scale, as shown below:\n>\n>     | Method       | LLM Backbone | Training Steps | Training Time (per 10k) | Relative Training Cost |\n>    | ------------ | ------------ | -------------- | ----------------------- | ---------------------- |\n>     | $\\pi_0$      | 2.6B         | 800–1200k      | 4.3 h                   | **~14.1×**             |\n>     | $\\pi_0$+FAST | 2.6B         | 80–200k        | 6.3 h                   | **~2.9×**              |\n>     | $\\pi_{0.5}$  | 2.6B         | -              | -                       | -                      |\n>     | OpenVLA-OFT  | 7B           | 50–150k        | 12.5 h                  | **~4.0×**              |\n>     | **CogVLA**   | **7B**       | **50–80k**     | **4.7 h**               | **1 (baseline)**       |\n> \n>     Notes:\n>     (i) We reproduced the $\\pi$ series based on the open-source `openpi` codebase.\n>      (ii) All models were trained on 4× A800 (80GB) GPUs.\n>      (iii) Due to the unavailability of $\\pi_{0.5}$ code, we could not include its training evaluation.\n> \n> - **Model Capability**:\n>\n>     Beyond training efficiency, CogVLA also demonstrates strong capability—especially on long-horizon tasks such as LIBERO-Long—where its performance is substantially better than the $\\pi$ series:\n>\n>     | Method                               | LIBERO-Spatial | LIBERO-Object | LIBERO-Goal | LIBERO-Long      | LIBERO-Overall |\n>    | ------------------------------------ | -------------- | ------------- | ----------- | ---------------- | -------------- |\n>     | $\\pi_0$                              | 96.8           | **98.8**      | 95.8        | 85.2 **(−10.2)** | 94.2           |\n>     | $\\pi_0$+FAST                         | 96.4           | 96.8          | 88.6        | 60.2 **(−35.4)** | 85.5           |\n>     | $\\pi_{0.5}$  (from scratch)          | 96.6           | 97.2          | 94.6        | 84.8 **(−10.4)** | 92.7           |\n>     | $\\pi_{0.5}$  (from generalist model) | 98.0           | 97.8          | 95.6        | 85.8 **(−9.4)**  | 96.0           |\n>     | **CogVLA**                           | **98.6**       | **98.8**      | **96.6**    | **95.4**         | **97.4**       |\n> \n> - We will include this comparative analysis in the main table of our revised submission to more comprehensively showcase CogVLA’s advantages over state-of-the-art baselines.\n>\n> **(2) Gemini Robotics**\n>\n> - Gemini Robotics is not open-sourced and primarily evaluated on the internal Gemini Robotics Benchmark defined by Google DeepMind, which we currently cannot access or reproduce. Therefore, to provide a meaningful comparison, we followed the task definitions from the Gemini paper and implemented them on the ALOHA platform. CogVLA achieves higher accuracy, particularly on tasks where Gemini Robotics shows suboptimal performance:\n>\n>     |                 | Task 1   | Task 2   | Task 3    |\n>    | --------------- | -------- | -------- | --------- |\n>     | Gemini Robotics | 2/10     | 5/10     | 10/10     |\n>     | $\\pi_0$         | 0/10     | 2/10     | 9/10      |\n>     | **CogVLA**      | **4/10** | **6/10** | **10/10** |\n> \n>     Task definitions: Task 1 – Unfold the mat; Task 2 – Fold the blue T-shirt from bottom to top; Task 3 – Close the laptop.\n>\n> - Additional experiments will be conducted upon receiving access via Google Trusted Tester evaluation, and the results will be incorporated in future updates.\n\nWe sincerely thank you for the insightful suggestions, which have been invaluable in refining the paper’s structure and empirical support. We hope our responses sufficiently address your concerns."}]

Author rebuttal:

Meta-review: This is a solid paper that introduces CogVLA, a cognition-aligned framework for vision-language-action tasks. The main strengths highlighted by reviewers are the novelty of the design, the strong efficiency–performance trade-off, and the clarity of the presentation. 

The main concerns raised were about (i) how CAtten differs from π₀, (ii) how the approach compares to existing VLM-based methods, (iii) whether the baselines were sufficient, and (iv) the reliability of results. The authors provided an excellent rebuttal, clarifying conceptual differences, adding stronger baselines, and reporting additional trials with standard deviations. These additions directly addressed the reviewers’ questions and helped build confidence in the contribution.

After the rebuttal, all reviewers agreed the concerns were resolved and maintained positive scores (5, 4, 4, 4). Overall, the paper makes a novel and significant contribution, and given the strong rebuttal and consensus, I recommend acceptance.
Decision: Accept (poster)