Generalizable Humanoid Manipulation with Improved 3D Diffusion Policies
Authors: name: Yanjie Ze; name: Zixuan Chen; name: Wenhao Wang; name: Tianyi Chen; name: Xialin He; name: Ying Yuan; name: Xue Bin Peng; name: Jiajun Wu
Published: 2024-10-14 17:59:00

Abstract: Humanoid robots capable of autonomous operation in diverse environments have
long been a goal for roboticists. However, autonomous manipulation by humanoid
robots has largely been restricted to one specific scene, primarily due to the
difficulty of acquiring generalizable skills. Recent advances in 3D visuomotor
policies, such as the 3D Diffusion Policy (DP3), have shown promise in
extending these capabilities to wilder environments. However, 3D visuomotor
policies often rely on camera calibration and point-cloud segmentation, which
present challenges for deployment on mobile robots like humanoids. In this
work, we introduce the Improved 3D Diffusion Policy (iDP3), a novel 3D
visuomotor policy that eliminates these constraints by leveraging egocentric 3D
visual representations. We demonstrate that iDP3 enables a full-sized humanoid
robot to autonomously perform skills in diverse real-world scenarios, using
only data collected in the lab. Videos are available at:
https://humanoid-manipulation.github.io