Title: Rewiring Experts on the Fly: Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
Primary area: foundation or frontier models, including LLMs
Keywords: LLM, MOE, Testtime

Abstract: Mixture-of-Experts (MoE) models achieve efficient scaling through sparse expert activation, but often suffer from suboptimal routing decisions that create performance bottlenecks during inference due to distribution shifts between training and test data. While existing test-time adaptation methods could potentially address these routing issues, they primarily focus on dense models and require access to external data, thereby limiting their practical applicability to MoE architectures. We propose a data-free, online test-time rerouting framework that optimizes MoE expert selection using only the input prompt itself. Our method operates through a continuous optimization approach that first adapts routing based on the input prompt, then maintains this adaptation throughout the generation process. We implement this through lightweight additive vectors that update only router logits in selected layers, maintaining computational efficiency while preventing over-adaptation. Experimental results show consistent performance gains on challenging reasoning tasks. For example, our method achieves a 5.5\% improvement on HumanEval with OLMOE. Furthermore, the approach combines effectively with other test-time techniques, producing 6\% average improvements when integrated with self-consistency on DeepSeek-V2-Lite.