Title: Fine-Grained Mixture of Experts for Medical Multimodal Learning
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Mixture of Experts (MoE), Fine-Grained Experts, Medical Multimodal Learning, Adaptive Expert Grouping

Abstract: Fine-Grained Mixture of Experts is a powerful architecture for scaling large models, yet its application in specialized domains like medicine remains underexplored. In this work, we conduct the first systematic study of expert granularity in a medical multimodal context. Our findings reveal a fundamental trade-off: while increasing granularity significantly enhances out-of-distribution (OOD) generalization and robustness, it slightly degrades in-distribution (ID) fitting and, critically, creates substantial functional redundancy among experts. We argue that this redundancy imposes unnecessary computational pressure on the routing mechanism. To address this, we introduce Adaptive Expert Grouping (AEG), a novel, end-to-end learnable mechanism that exploits this redundancy by dynamically clustering functionally similar experts. By shifting routing decisions from the individual expert level to the group level, AEG substantially reduces computational overhead and improves model sparsity, while preserving the generalization benefits of the fine-grained architecture. Our work offers a new path towards building more efficient and robust MoE models for specialized domains.

TL;DR: We find that fine-graining MoE experts boosts OOD generalization in medical AI and propose Adaptive Expert Grouping (AEG) to exploit the emergent functional redundancy for more efficient routing.