MotionStreamer: Streaming Motion Generation via Diffusion-based
  Autoregressive Model in Causal Latent Space
Authors: name: Lixing Xiao, name: Shunlin Lu, name: Huaijin Pi, name: Ke Fan, name: Liang Pan, name: Yueer Zhou, name: Ziyong Feng, name: Xiaowei Zhou, name: Sida Peng, name: Jingbo Wang

This paper addresses the challenge of text-conditioned streaming motion
generation, which requires us to predict the next-step human pose based on
variable-length historical motions and incoming texts. Existing methods
struggle to achieve streaming motion generation, e.g., diffusion models are
constrained by pre-defined motion lengths, while GPT-based methods suffer from
delayed response and error accumulation problem due to discretized non-causal
tokenization. To solve these problems, we propose MotionStreamer, a novel
framework that incorporates a continuous causal latent space into a
probabilistic autoregressive model. The continuous latents mitigate information
loss caused by discretization and effectively reduce error accumulation during
long-term autoregressive generation. In addition, by establishing temporal
causal dependencies between current and historical motion latents, our model
fully utilizes the available information to achieve accurate online motion
decoding. Experiments show that our method outperforms existing approaches
while offering more applications, including multi-round generation, long-term
generation, and dynamic motion composition. Project Page:
https://zju3dv.github.io/MotionStreamer/