H^{3}DP: Triply-Hierarchical Diffusion Policy for Visuomotor
  Learning
Authors: name: Yiyang Lu; name: Yufeng Tian; name: Zhecheng Yuan; name: Xianbang Wang; name: Pu Hua; name: Zhengrong Xue; name: Huazhe Xu
Published: 2025-05-12 17:59:43

Abstract: Visuomotor policy learning has witnessed substantial progress in robotic
manipulation, with recent approaches predominantly relying on generative models
to model the action distribution. However, these methods often overlook the
critical coupling between visual perception and action prediction. In this
work, we introduce Triply-Hierarchical Diffusion
Policy~(H^{\mathbf{3}DP}), a novel visuomotor learning framework
that explicitly incorporates hierarchical structures to strengthen the
integration between visual features and action generation. H^{3}DP contains
3 levels of hierarchy: (1) depth-aware input layering that organizes
RGB-D observations based on depth information; (2) multi-scale visual
representations that encode semantic features at varying levels of granularity;
and (3) a hierarchically conditioned diffusion process that aligns the
generation of coarse-to-fine actions with corresponding visual features.
Extensive experiments demonstrate that H^{3}DP yields a +27.5%
average relative improvement over baselines across 44 simulation
tasks and achieves superior performance in 4 challenging bimanual
real-world manipulation tasks. Project Page: https://lyy-iiis.github.io/h3dp/.