Parallelized Autoregressive Visual Generation
Authors: name: Yuqing Wang; name: Shuhuai Ren; name: Zhijie Lin; name: Yujin Han; name: Haoyuan Guo; name: Zhenheng Yang; name: Difan Zou; name: Jiashi Feng; name: Xihui Liu
Published: 2024-12-19 17:59:54

Abstract: Autoregressive models have emerged as a powerful approach for visual
generation but suffer from slow inference speed due to their sequential
token-by-token prediction process. In this paper, we propose a simple yet
effective approach for parallelized autoregressive visual generation that
improves generation efficiency while preserving the advantages of
autoregressive modeling. Our key insight is that parallel generation depends on
visual token dependencies-tokens with weak dependencies can be generated in
parallel, while strongly dependent adjacent tokens are difficult to generate
together, as their independent sampling may lead to inconsistencies. Based on
this observation, we develop a parallel generation strategy that generates
distant tokens with weak dependencies in parallel while maintaining sequential
generation for strongly dependent local tokens. Our approach can be seamlessly
integrated into standard autoregressive models without modifying the
architecture or tokenizer. Experiments on ImageNet and UCF-101 demonstrate that
our method achieves a 3.6x speedup with comparable quality and up to 9.5x
speedup with minimal quality degradation across both image and video generation
tasks. We hope this work will inspire future research in efficient visual
generation and unified autoregressive modeling. Project page:
https://epiphqny.github.io/PAR-project.