Ming-Omni: A Unified Multimodal Model for Perception and Generation
Authors: name: Inclusion AI, name: Biao Gong, name: Cheng Zou, name: Chuanyang Zheng, name: Chunluan Zhou, name: Canxiang Yan, name: Chunxiang Jin, name: Chunjie Shen, name: Dandan Zheng, name: Fudong Wang, name: Furong Xu, name: GuangMing Yao, name: Jun Zhou, name: Jingdong Chen, name: Jianxin Sun, name: Jiajia Liu, name: Jianjiang Zhu, name: Jun Peng, name: Kaixiang Ji, name: Kaiyou Song, name: Kaimeng Ren, name: Libin Wang, name: Lixiang Ru, name: Lele Xie, name: Longhua Tan, name: Lyuxin Xue, name: Lan Wang, name: Mochen Bai, name: Ning Gao, name: Pei Chen, name: Qingpei Guo, name: Qinglong Zhang, name: Qiang Xu, name: Rui Liu, name: Ruijie Xiong, name: Sirui Gao, name: Tinghao Liu, name: Taisong Li, name: Weilong Chai, name: Xinyu Xiao, name: Xiaomei Wang, name: Xiaoxue Chen, name: Xiao Lu, name: Xiaoyu Li, name: Xingning Dong, name: Xuzheng Yu, name: Yi Yuan, name: Yuting Gao, name: Yunxiao Sun, name: Yipeng Chen, name: Yifei Wu, name: Yongjie Lyu, name: Ziping Ma, name: Zipeng Feng, name: Zhijiang Fang, name: Zhihao Qiu, name: Ziyuan Huang, name: Zhengyu He

We propose Ming-Omni, a unified multimodal model capable of processing
images, text, audio, and video, while demonstrating strong proficiency in both
speech and image generation. Ming-Omni employs dedicated encoders to extract
tokens from different modalities, which are then processed by Ling, an MoE
architecture equipped with newly proposed modality-specific routers. This
design enables a single model to efficiently process and fuse multimodal inputs
within a unified framework, thereby facilitating diverse tasks without
requiring separate models, task-specific fine-tuning, or structural redesign.
Importantly, Ming-Omni extends beyond conventional multimodal models by
supporting audio and image generation. This is achieved through the integration
of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for
high-quality image generation, which also allow the model to engage in
context-aware chatting, perform text-to-speech conversion, and conduct
versatile image editing. Our experimental results showcase Ming-Omni offers a
powerful solution for unified perception and generation across all modalities.
Notably, our proposed Ming-Omni is the first open-source model we are aware of
to match GPT-4o in modality support, and we release all code and model weights
to encourage further research and development in the community.