Title: Draft-and-Target Sampling for Video Generation Policy
Primary area: applications to robotics, autonomy, planning
Keywords: Video Generation Policy; Video Generation for Robotics; Speculative Decoding; Inference Acceleration

Abstract: Video generation models have been used as a robot policy to predict the future states of executing a task conditioned on task description and observation. Previous works ignore their high computational cost and long inference time. To address this challenge, we propose Draft-and-Target Sampling, a novel speculative decoding-like inference paradigm for video generation policy that is training-free and can improve inference efficiency. We modify the classic principle of speculative decoding design and redefine the draft and target as two complementary denoising trajectories. To further speedup generation, we introduce token chunking and progressive acceptance strategy to reduce redundant computation. Experiments on three benchmarks show that our method can achieve up to 2.1x speedup and improve the efficiency of current state-of-the-art methods with minimal compromise to the success rate. Our code is available at anonymous github.

TL;DR: We propose a speculative decoding-like and training-free framework to accelerate video generation policies,  our method achieves up to 2.1x speedup with minimal compromise to the success rate.