DexGraspVLA: A Vision-Language-Action Framework Towards General
  Dexterous Grasping
Authors: name: Yifan Zhong; name: Xuchuan Huang; name: Ruochong Li; name: Ceyao Zhang; name: Yitao Liang; name: Yaodong Yang; name: Yuanpei Chen
Published: 2025-02-28 09:57:20

Abstract: Dexterous grasping remains a fundamental yet challenging problem in robotics.
A general-purpose robot must be capable of grasping diverse objects in
arbitrary scenarios. However, existing research typically relies on specific
assumptions, such as single-object settings or limited environments, leading to
constrained generalization. Our solution is DexGraspVLA, a hierarchical
framework that utilizes a pre-trained Vision-Language model as the high-level
task planner and learns a diffusion-based policy as the low-level Action
controller. The key insight lies in iteratively transforming diverse language
and visual inputs into domain-invariant representations, where imitation
learning can be effectively applied due to the alleviation of domain shift.
Thus, it enables robust generalization across a wide range of real-world
scenarios. Notably, our method achieves a 90+% success rate under thousands of
unseen object, lighting, and background combinations in a ``zero-shot''
environment. Empirical analysis further confirms the consistency of internal
model behavior across environmental variations, thereby validating our design
and explaining its generalization performance. We hope our work can be a step
forward in achieving general dexterous grasping. Our demo and code can be found
at https://dexgraspvla.github.io/.