Mixture of States: Routing Token-Level Dynamics for Multimodal Generation
Authors: name: Haozhe Liu; name: Ding Liu; name: Mingchen Zhuge; name: Zijian Zhou; name: Tian Xie; name: Sen He; name: Yukang Yang; name: Shuming Liu; name: Yuren Cong; name: Jiadong Guo; name: Hongyu Xu; name: Ke Xu; name: Kam-Woh Ng; name: Juan C. Pérez; name: Juan-Manuel~Pérez-Rúa; name: Tao Xiang; name: Wei Liu; name: Shikun Liu; name: Jürgen Schmidhuber
Published: 2025-11-15 13:24:57

Abstract: We introduce MoS (Mixture of States), a novel fusion paradigm for multimodal diffusion models that merges modalities using flexible, state-based interactions. The core of MoS is a learnable, token-wise router that creates denoising timestep- and input-dependent interactions between modalities' hidden states, precisely aligning token-level features with the diffusion trajectory. This router sparsely selects the top-k hidden states and is trained with an ε-greedy strategy, efficiently selecting contextual features with minimal learnable parameters and negligible computational overhead. We validate our design with text-to-image generation (MoS-Image) and editing (MoS-Editing), which achieve state-of-the-art results. With only 3B to 5B parameters, our models match or surpass counterparts up to 4times larger. These findings establish MoS as a flexible and compute-efficient paradigm for scaling multimodal diffusion models.