Emerging Properties in Unified Multimodal Pretraining
Authors: name: Chaorui Deng, name: Deyao Zhu, name: Kunchang Li, name: Chenhui Gou, name: Feng Li, name: Zeyu Wang, name: Shu Zhong, name: Weihao Yu, name: Xiaonan Nie, name: Ziang Song, name: Guang Shi, name: Haoqi Fan

Unifying multimodal understanding and generation has shown impressive
capabilities in cutting-edge proprietary systems. In this work, we introduce
BAGEL, an open0source foundational model that natively supports multimodal
understanding and generation. BAGEL is a unified, decoder0only model pretrained
on trillions of tokens curated from large0scale interleaved text, image, video,
and web data. When scaled with such diverse multimodal interleaved data, BAGEL
exhibits emerging capabilities in complex multimodal reasoning. As a result, it
significantly outperforms open-source unified models in both multimodal
generation and understanding across standard benchmarks, while exhibiting
advanced multimodal reasoning abilities such as free-form image manipulation,
future frame prediction, 3D manipulation, and world navigation. In the hope of
facilitating further opportunities for multimodal research, we share the key
findings, pretraining details, data creation protocal, and release our code and
checkpoints to the community. The project page is at https://bagel-ai.org/