PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping
  with VLM
Authors: name: Alan Dao; name: Dinh Bach Vu; name: Tuan Le Duc Anh; name: Bui Quang Huy
Published: 2025-03-10 09:34:05

Abstract: This paper introduces PoseLess, a novel framework for robot hand control that
eliminates the need for explicit pose estimation by directly mapping 2D images
to joint angles using projected representations. Our approach leverages
synthetic training data generated through randomized joint configurations,
enabling zero-shot generalization to real-world scenarios and cross-morphology
transfer from robotic to human hands. By projecting visual inputs and employing
a transformer-based decoder, PoseLess achieves robust, low-latency control
while addressing challenges such as depth ambiguity and data scarcity.
Experimental results demonstrate competitive performance in joint angle
prediction accuracy without relying on any human-labelled dataset.