DynaGuide: Steering Diffusion Polices with Active Dynamic Guidance
Authors: name: Maximilian Du, name: Shuran Song

Deploying large, complex policies in the real world requires the ability to
steer them to fit the needs of a situation. Most common steering approaches,
like goal-conditioning, require training the robot policy with a distribution
of test-time objectives in mind. To overcome this limitation, we present
DynaGuide, a steering method for diffusion policies using guidance from an
external dynamics model during the diffusion denoising process. DynaGuide
separates the dynamics model from the base policy, which gives it multiple
advantages, including the ability to steer towards multiple objectives, enhance
underrepresented base policy behaviors, and maintain robustness on low-quality
objectives. The separate guidance signal also allows DynaGuide to work with
off-the-shelf pretrained diffusion policies. We demonstrate the performance and
features of DynaGuide against other steering approaches in a series of
simulated and real experiments, showing an average steering success of 70% on a
set of articulated CALVIN tasks and outperforming goal-conditioning by 5.4x
when steered with low-quality objectives. We also successfully steer an
off-the-shelf real robot policy to express preference for particular objects
and even create novel behavior. Videos and more can be found on the project
website: https://dynaguide.github.io