NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation
Authors: name: Huichao Zhang, name: Liao Qu, name: Yiheng Liu, name: Hang Chen, name: Yangyang Song, name: Yongsheng Dong, name: Shikun Sun, name: Xian Li, name: Xu Wang, name: Yi Jiang, name: Hu Ye, name: Bo Chen, name: Yiming Gao, name: Peng Liu, name: Akide Liu, name: Zhipeng Yang, name: Qili Deng, name: Linjie Xing, name: Jiyang Liu, name: Zhao Wang, name: Yang Zhou, name: Mingcong Liu, name: Yi Zhang, name: Qian He, name: Xiwei Hu, name: Zhongqi Qi, name: Jie Shao, name: Zhiye Fu, name: Shuai Wang, name: Fangmin Chen, name: Xuezhi Chai, name: Zhihua Wu, name: Yitong Wang, name: Zehuan Yuan, name: Daniel K. Du, name: Xinglong Wu
Organization: ByteDance

We present NextFlow, a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image discrete tokens. By leveraging a unified vision representation within a unified autoregressive architecture, NextFlow natively activates multimodal understanding and generation capabilities, unlocking abilities of image editing, interleaved content and video generation. Motivated by the distinct nature of modalities - where text is strictly sequential and images are inherently hierarchical - we retain next-token prediction for text but adopt next-scale prediction for visual generation. This departs from traditional raster-scan methods, enabling the generation of 1024x1024 images in just 5 seconds - orders of magnitude faster than comparable AR models. We address the instabilities of multi-scale generation through a robust training recipe. Furthermore, we introduce a prefix-tuning strategy for reinforcement learning. Experiments demonstrate that NextFlow achieves state-of-the-art performance among unified models and rivals specialized diffusion baselines in visual quality.