Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs
  at Test Time
Authors: name: Yixuan Han; name: Fan Ma; name: Ruijie Quan; name: Yi Yang
Published: 2025-09-26 16:49:10

Abstract: Test-Time Scaling (TTS) enhances the reasoning ability of large language
models (LLMs) by allocating additional computation during inference. However,
existing approaches primarily rely on output-level sampling while overlooking
the role of model architecture. In mainstream Mixture-of-Experts (MoE) LLMs, we
observe that varying the number of activated experts yields complementary
solution sets with stable accuracy, revealing a new and underexplored source of
diversity. Motivated by this observation, we propose Dynamic Experts Search
(DES), a TTS strategy that elevates expert activation into a controllable
dimension of the search space. DES integrates two key components: (1) Dynamic
MoE, which enables direct control of expert counts during inference to generate
diverse reasoning trajectories without additional cost; and (2) Expert
Configuration Inheritance, which preserves consistent expert counts within a
reasoning path while varying them across runs, thereby balancing stability and
diversity throughout the search. Extensive experiments across MoE
architectures, verifiers and reasoning benchmarks (i.e., math, code and
knowledge) demonstrate that DES reliably outperforms TTS baselines, enhancing
accuracy and stability without additional cost. These results highlight DES as
a practical and scalable form of architecture-aware TTS, illustrating how
structural flexibility in modern LLMs can advance reasoning.