JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified
  Multimodal Understanding and Generation
Authors: name: Yiyang Ma; name: Xingchao Liu; name: Xiaokang Chen; name: Wen Liu; name: Chengyue Wu; name: Zhiyu Wu; name: Zizheng Pan; name: Zhenda Xie; name: Haowei Zhang; name: Xingkai yu; name: Liang Zhao; name: Yisong Wang; name: Jiaying Liu; name: Chong Ruan
Published: 2024-11-12 17:55:10

Abstract: We present JanusFlow, a powerful framework that unifies image understanding
and generation in a single model. JanusFlow introduces a minimalist
architecture that integrates autoregressive language models with rectified
flow, a state-of-the-art method in generative modeling. Our key finding
demonstrates that rectified flow can be straightforwardly trained within the
large language model framework, eliminating the need for complex architectural
modifications. To further improve the performance of our unified model, we
adopt two key strategies: (i) decoupling the understanding and generation
encoders, and (ii) aligning their representations during unified training.
Extensive experiments show that JanusFlow achieves comparable or superior
performance to specialized models in their respective domains, while
significantly outperforming existing unified approaches across standard
benchmarks. This work represents a step toward more efficient and versatile
vision-language models.