Chain-of-Experts: Unlocking the Communication Power of
  Mixture-of-Experts Models
Authors: name: Zihan Wang; name: Rui Pan; name: Jiarui Yao; name: Robert Csordas; name: Linjie Li; name: Lu Yin; name: Jiajun Wu; name: Tong Zhang; name: Manling Li; name: Shiwei Liu
Published: 2025-06-23 02:15:43

Abstract: We propose Chain-of-Experts (CoE), a new Mixture-of-Experts (MoE)
architecture that introduces sequential expert communication within each layer.
Unlike traditional MoE models, where experts operate independently in parallel,
CoE processes tokens iteratively across a chain of experts inside a layer. To
support dynamic expert selection across iterations, CoE employs a dedicated
router at each iteration step within a layer. This design allows tokens to
re-evaluate and select different experts during each iteration, rather than
being statically assigned. As a result, CoE introduces a flexible routing
mechanism that increases the diversity of expert combinations and enriches the
model's representational capacity. CoE demonstrates improved performance under
fixed compute: on math reasoning tasks, it reduces validation loss from 1.20 to
1.12 compared to a standard MoE. Beyond performance, CoE offers a new scaling
axis: depth through expert iteration, which complements conventional
width/depth scaling. For example, using 2x iterations matches the performance
of 3x expert selections (in width), while reducing memory usage by 17.6-42%
relative to other scaling strategies. Our analysis reveals that CoE's benefits
stem from its iterative residual structure and enhanced expert specialization
empowered by iterative routing, which together unlock more expressive
representations. Code is available at https://github.com/ZihanWang314/coe.