Title: EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Efficient AI

Embodied AI

Vision-Language-Action Model
Abstract: Vision-Language-Action (VLA) models, particularly diffusion-based architectures, demonstrate transformative potential for embodied intelligence but are severely hampered by high computational and memory demands stemming from extensive inherent and inference-time redundancies. While existing acceleration efforts often target isolated inefficiencies, such piecemeal solutions typically fail to holistically address the varied computational and memory bottlenecks across the entire VLA pipeline, thereby limiting practical deployability.  We introduce EfficientVLA, a structured and training-free inference acceleration framework that systematically eliminates these barriers by cohesively exploiting multifaceted redundancies. EfficientVLA synergistically integrates three targeted strategies: (1) pruning of functionally inconsequential layers from the language module, guided by an analysis of inter-layer redundancies; (2) optimizing the visual processing pathway through a task-aware strategy that selects a compact, diverse set of visual tokens, balancing task-criticality with informational coverage; and (3) alleviating temporal computational redundancy within the iterative diffusion-based action head by strategically caching and reusing key intermediate features.
We apply our method to a standard VLA model CogACT, yielding a $1.93\times$ inference speedup and reduces FLOPs to 28.9%, with only a 0.6% success rate drop in the SIMPLER benchmark.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to improve the inference efficiency of pretrained VLA models and introduces a training-free framework named EfficientVLA. The proposed efficiency improvements are achieved through three key techniques: selective vision token pruning, LLM layer pruning, and action head computation caching. Compared to the vanilla model, EfficientVLA achieves a 1.93× inference speedup under the SIMPLER setting, with only a 0.6% drop in performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The redundancy analysis presented in Figure 1 provides valuable insight into the computational inefficiency in VLA models.\n2. The proposed framework achieves state-of-the-art inference efficiency with minimal performance degradation, demonstrating the effectiveness of combining the three techniques.\n\n**Weaknesses**\n1. The framework is only evaluated on the CogACT model. Experiments on additional VLA architectures would strengthen the generalizability and robustness of the method.\n2. More ablations on visual token reduction are needed. For instance, analyzing the individual contributions of Vkey, Vtask, and Vdiv would clarify their roles. Additionally, visualizations of the token pruning process would improve interpretability.\n\n**Minor**\n1. It seems that $r_i$ in Equation (2) and $s_i$ in Section 3.3.2 refer to the same variable. Please clarify or correct if this is a typo.\n2. In Table 4, is the ratio shown the token reduction ratio or the token preservation ratio? Also, why is the last column showing a 100% ratio while its FLOPs are the highest among all columns?\n\nQUESTIONS:\n1. Based on Table 5, visual token pruning appears to improve visual success rate, whereas model compression and action cache mainly reduce computational costs. Could the authors elaborate on this finding and provide further discussion?\n2. Regarding the action head caching, could the authors clarify how feature updates are handled when self-attention and MLP layers are bypassed? An explicit explanation of the feature flow in this case would greatly help reader comprehension.\n\nLIMITATIONS:\nWhile the technical novelty of the proposed methods is limited, since similar techniques have been explored in the VLM or LLM domains, their integration into the VLA context is also important. The paper effectively adapts existing ideas to a new domain, offering meaningful efficiency improvements.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents EfficientVLA, a training-free, structured inference acceleration framework for diffusion-based Vision-Language-Action (VLA) models. The method systematically addresses inefficiencies across the VLA pipeline via three key components: (1) non-contiguous pruning of redundant layers in the language module based on inter-layer similarity; (2) task-aware and diversity-promoting visual token pruning; and (3) temporal feature caching within the diffusion-based action head. Experiments on the CogACT model in the SIMPLER environment show up to 1.93× inference speedup and 71.1% FLOPs reduction with negligible accuracy loss (<0.6%).\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The paper is well-motivated, with a bottleneck analysis (LLM memory, visual redundancy, and diffusion-step redundancy) that justifies a holistic acceleration framework.\n\n- EfficientVLA is training-free, making it easily deployable across existing VLA models without retraining.\n\n- The method effectively reduces redundancies across the VLA pipeline by combining layer pruning, task-relevant and diverse visual token selection, and feature caching in diffusion-based action heads.\n\n**Weakness:**\n\n- More experiments are encouraged to validate the generalizability of the proposed method, especially on other VLA models and dataset such as Libro. Furthermore, the SIMPLER benchmark includes only four tasks, limiting the evaluation's diversity.\n\n- Real-world robotic experiments would strengthen the claims regarding deployment efficiency, especially in terms of latency and robustness under hardware constraints.\n\n- It is encouraged to include related works with respect to existing techniques in layer pruning, token pruning, and temporal caching.\n\nQUESTIONS:\n- Do redundancies in different modules interact with each other? For example, does pruning in the language module influence token selection or the effectiveness of diffusion caching? Is there a trade-off involved when reducing redundancy across different stages of the VLA pipeline?\n\n- Can EfficientVLA generalize to other multimodal tasks beyond robotics? In particular, could the techniques for visual token pruning and temporal caching be applied to VQA, image captioning, or other vision-language domains?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates methods to accelerate the inference speed of vision-language-action (VLA) models using structured and training-free approaches. It employs three techniques—pruning, dynamic visual token selection, and caching/reusing key features—to achieve acceleration. The proposed approach outperforms CogACT and other baselines on the SIMPLER benchmark.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The investigated problem is significant and relevant. Employing a training-free acceleration method for VLA models is meaningful, particularly given that speed is crucial in robot learning.\n2. The proposed method is reasonable, and the paper is clearly written and easy to follow.\n3. The analysis presented is insightful and intuitive.\n4. The ablation study conducted is thorough and comprehensive. Code availability facilitates reproducibility.\n\nWeaknesses:\n\n1. CogACT is not a representative VLA model. Its baseline performance is low, making it less ideal for this study. A model like pi0 would have been a more suitable choice.\n\n2. Evaluating robot learning solely based on task completion overlooks other crucial aspects, such as motion smoothness. Does EfficientVLA predict the 6D pose or joint positions in the SIMPLER environment? If the output changes, can EfficientVLA maintain smoothness?\n\n3. FLOPs and parameter counts are not the most critical metrics for evaluating model efficiency in robotics. A more relevant metric would be how many actions the model can predict per second given the same chunk size.\n\n4. A comparison with Otter [1], a closely related study that also leverages CLIP for visual token reduction, is necessary.\n\n[1] Otter: A vision-language-action model with text-aware visual feature extraction.\n\nQUESTIONS:\nAdditional experiments using pi0 would strengthen the evaluation. A comparison with Otter regarding visual token pruning is recommended. Conducting real-world robotic experiments would also enhance the validation of the approach.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}