Ming-Omni: A Unified Multimodal Model for Perception and Generation
Authors: name: Inclusion AI; name: Biao Gong; name: Cheng Zou; name: Chuanyang Zheng; name: Chunluan Zhou; name: Canxiang Yan; name: Chunxiang Jin; name: Chunjie Shen; name: Dandan Zheng; name: Fudong Wang; name: Furong Xu; name: GuangMing Yao; name: Jun Zhou; name: Jingdong Chen; name: Jianxin Sun; name: Jiajia Liu; name: Jianjiang Zhu; name: Jun Peng; name: Kaixiang Ji; name: Kaiyou Song; name: Kaimeng Ren; name: Libin Wang; name: Lixiang Ru; name: Lele Xie; name: Longhua Tan; name: Lyuxin Xue; name: Lan Wang; name: Mochen Bai; name: Ning Gao; name: Pei Chen; name: Qingpei Guo; name: Qinglong Zhang; name: Qiang Xu; name: Rui Liu; name: Ruijie Xiong; name: Sirui Gao; name: Tinghao Liu; name: Taisong Li; name: Weilong Chai; name: Xinyu Xiao; name: Xiaomei Wang; name: Xiaoxue Chen; name: Xiao Lu; name: Xiaoyu Li; name: Xingning Dong; name: Xuzheng Yu; name: Yi Yuan; name: Yuting Gao; name: Yunxiao Sun; name: Yipeng Chen; name: Yifei Wu; name: Yongjie Lyu; name: Ziping Ma; name: Zipeng Feng; name: Zhijiang Fang; name: Zhihao Qiu; name: Ziyuan Huang; name: Zhengyu He
Published: 2025-06-11 02:50:49

Abstract: We propose Ming-Omni, a unified multimodal model capable of processing
images, text, audio, and video, while demonstrating strong proficiency in both
speech and image generation. Ming-Omni employs dedicated encoders to extract
tokens from different modalities, which are then processed by Ling, an MoE
architecture equipped with newly proposed modality-specific routers. This
design enables a single model to efficiently process and fuse multimodal inputs
within a unified framework, thereby facilitating diverse tasks without
requiring separate models, task-specific fine-tuning, or structural redesign.
Importantly, Ming-Omni extends beyond conventional multimodal models by
supporting audio and image generation. This is achieved through the integration
of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for
high-quality image generation, which also allow the model to engage in
context-aware chatting, perform text-to-speech conversion, and conduct
versatile image editing. Our experimental results showcase Ming-Omni offers a
powerful solution for unified perception and generation across all modalities.
Notably, our proposed Ming-Omni is the first open-source model we are aware of
to match GPT-4o in modality support, and we release all code and model weights
to encourage further research and development in the community.