Title: Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training
Venue: NeurIPS 2025
Accepted: yes
Keywords: MoE

cognitive experts

thinking

large reasoning model
Abstract: Mixture-of-Experts (MoE) architectures within Large Reasoning Models (LRMs) have achieved impressive reasoning capabilities by selectively activating experts to facilitate structured cognitive processes. Despite notable advances, existing reasoning models often suffer from cognitive inefficiencies like overthinking and underthinking. To address these limitations, we introduce a novel inference-time steering methodology called Reinforcing Cognitive Experts (RICE), designed to improve reasoning depth and efficiency without additional training or complex heuristics. Leveraging normalized Pointwise Mutual Information (nPMI), we systematically identify specialized experts, termed cognitive experts that orchestrate meta-level reasoning operations characterized by tokens like <think>. Empirical evaluations with leading MoE-based LRMs (DeepSeek-R1 and Qwen3-235B) on rigorous quantitative and scientific reasoning benchmarks (AIME and GPQA Diamond) demonstrate noticeable and consistent improvements in reasoning accuracy, cognitive efficiency, and cross-domain generalization. Crucially, our lightweight approach substantially outperforms prevalent reasoning-steering techniques, such as prompt design and decoding constraints, while preserving the model's general instruction-following skills. These results highlight reinforcing cognitive experts as a promising, practical, and interpretable direction to enhance cognitive efficiency within advanced reasoning models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies Mixture-of-Experts (MoE) models and proposes to identify which experts are most associated with “thinking” behavior. To achieve this, the authors use a mutual information metric measuring how strongly each expert activates on <think> </think> tokens compared to other tokens. They then demonstrate that stimulating (via steering) these identified experts can yield performance gains on reasoning tasks.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n\nThis is an enjoyable paper to read, with a clear, simple idea and a good execution. The finding that manipulating just a few experts can improve reasoning performance is interesting. The experimental setup is strong, covering different domains and testing generalization to another dataset.\n\n\n# Weaknesses:\n\nIn answering RQ1, the paper focuses on the top 5 experts. It is unclear why the number 5 was chosen over other possible thresholds. It would be much more informative to examine the entire distribution of normalized pointwise mutual information (nPMI) across experts. Is there a clear group of “thinking-specialized” experts that stand out with very high nPMI scores, or does the distribution decline more gradually?\n\nOverall, the experimental section has great potential, but the current structure and reporting fall somewhat short of fully realizing it. I think the research questions (RQs) could be structured and answered in a more logical and cumulative order to better support the paper’s core claims. For example:\n\n- RQ1: Are there “cognitive experts” specialized in thinking? That is, do a few experts exhibit peak nPMI on thinking tokens? This would require showing the distribution of nPMI over all experts, to see whether the top experts have a clear spike in nPMI scores or whether the distribution declines slowly.\n- RQ2: Do these experts differ across domains? (Current RQ2 / Table 1)\n- RQ3: Can we enhance performance by steering them? (Current RQ1 / Tables 2 and 3)\n- RQ4: Does stimulation of these experts generalize to other domains? (Current RQ2 / Table 4)\n- RQ5: Does stimulating these experts negatively impact general abilities? (Current RQ3 / Table 5)\n\nFinally, the paper does not report any uncertainty estimates alongside its results (e.g., standard deviations or confidence intervals). Including such measures is important for rigor.\n\nQUESTIONS:\n- What does the distribution of nPMI scores across experts look like? Is it sharply peaked for the top experts, or does it decrease more gradually?\n- Why does stimulating the “biology experts” fail to improve performance in the biology domain?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFor MoE models, this paper uses normalized pointwise mutual information to identify experts related to reasoning tokens, which are called cognitive experts. This paper proposes a method to improve the model's reasoning ability by increasing the weight of cognitive experts during routing. Experiments on some domains, especially math, show that the proposed method could improve the model's reasoning ability and outperform other training-free steering methods\n\nSTRENGTHS AND WEAKNESSES:\nThis finds an interesting phenomenon in MoE models, that is, the reasoning ability might be highly correlated with some specific experts. What's more, simply increasing the weights of these experts during routing could bring significant improvements in some math benchmarks. \n\nHowever, I think the experiments are not solid enough to support the authors' claim. Here are some of my concerns.\n\n1. The cognitive experts seem to be domain-specific. As shown in Table 2, the top-2 identified experts in Chemistry and Biology are different from those in Math and Physics. I wonder whether different datasets in the same domain would result in different cognitive experts? For example, will math-500 and AIME have the same cognitive experts?\n\n 2. The reinforcing method seems to only work in math, instead of in any other domains. As shown in Table 4, when reinforcing the cognitive experts in physics will make the performance in physics degrades from 91.9 to 89.5, and when reinforcing the cognitive experts in biology will make the performance in biology degrades from 79.0 to 73.9. The experiment in Chemistry is positive, but the improvement is not as significant as in the math domain. I also want to know the results of section 3.5 using other domains. Whether the proposed method still outperform other steering methods?\n\n3. Whether reinforcing cognitive experts could help the model to reason more effectively is in doubt. The experiment in Section 3.2 mainly supports this claim. However, as shown in Table 5, all models using the reinforce trick generate more tokens compared with the vanilla model. This contradicts the claim.\n\nQUESTIONS:\nWhether the proposed method is useful for all domains or only MATH? Please refer to the weaknesses for detailed comments.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper proposes a inference time model steering method for Mixture of Experts (MoE) models. They use pointwise mutual information to get a correlation between the generation of <think> token and activation of a particular expert in a MoE model. The experts which have a high mutual information are then considered as the ones contributing to the reasoning of the generated solution. The activation weight of such experts can be controlled to steer more reasoning and their results show that this helps with the end task on AIME and GPQA diamond datasets.

STRENGTHS AND WEAKNESSES:
Strengths:

1. Lightweight activation steering of models is an active area of research and such steering without training has broad applicability.
2. The paper is well written with clear experiments.
3. The steering method is interpretable and easy to control.
4. Results on math problems are encouraging.

Weaknesses:
1. The evaluation sets are small.
2. Results on math are clear and convincing, but results on other domains are not very significant and the writing does not call it out. For example from line 214 and 215: it is misleading to say 0.9% gain is moderate and 2.4% drop is minor.
3. Evaluation is done only on 2 models. 
4. It is not clear if the method generalizes to other reasoning tasks beyond QA.

QUESTIONS:
1. Could you provide insights into the distribution of the nPMI for experts?
2. There is a qualitative example of how reasoning changes. What are some other ways to evaluate if the reasoning/thinking tokens improved?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Reinforcing Cognitive Experts (RICE), an inference-time steering method for Mixture-of-Experts (MoE) large reasoning models (LRMs) to enhance reasoning depth and efficiency without additional training. Using normalized Pointwise Mutual Information (nPMI), the approach identifies \"cognitive experts\" highly correlated with reasoning markers, and then add weight on the identified experts. Experiments on DeepSeek-R1 and Qwen3-235B across mathematical (AIME) and scientific (GPQA Diamond) benchmarks show that reinforcing just the top two cognitive experts significantly improves reasoning accuracy, reduces token usage, and demonstrates cross-domain generalization to some extend.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The proposed RICE method demonstrates significant performance improvements across reasoning benchmarks.\n2. It is lightweight, requiring no additional training or complex heuristics.\n3. The paper features clear and well-structured writing, enhancing readability.\n\n\nWeakness:\n1. The results rely on only two benchmarks (AIME and GPQA Diamond) and two models (DeepSeek-R1 and Qwen3-235B), which limits the generalizability of the findings.\n2. As shown in Table 4, activating domain-specific experts does not consistently improve performance. For example, physics experts slightly degrade performance on physics tasks, and biology experts show reduced accuracy in biology. In contrast, math experts demonstrate the most robust effectiveness, highlighting potential domain-specific inconsistencies.\n3. Despite impressive performance gains, the paper lacks in-depth explanations for why RICE works. Visualizing and analyzing feature distributions or activation patterns after expert reweighting could provide crucial insights, requiring further analysis.\n4. What is the data for identifying the cognitive experts, and could there be some data leakage? \nTable 3 shows that the cognitive experts identified using AIME24 are applied to AIME25; however, the similarity between these datasets raises questions on data leakage. Could the author adopt some other benchmarks are evaluate the activated experts?\n5. eq(6): the expression \"t ∈ set Π\" should be \"x ∈ set Π\"?\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}