Title: ARM: Adaptive Reasoning Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Models

Large Reasoning Models

Adaptive Reasoning
Abstract: While large reasoning models demonstrate strong performance on complex tasks, they lack the ability to adjust reasoning token usage based on task difficulty. This often leads to the "overthinking" problem—excessive and unnecessary reasoning—which, although potentially mitigated by human intervention to control the token budget, still fundamentally contradicts the goal of achieving fully autonomous AI. In this work, we propose Adaptive Reasoning Model (ARM), a reasoning model capable of adaptively selecting appropriate reasoning formats based on the task at hand. These formats include three efficient ones—Direct Answer, Short CoT, and Code—as well as a more elaborate format, Long CoT. To train ARM, we introduce Ada-GRPO, an adaptation of Group Relative Policy Optimization (GRPO), which addresses the format collapse issue in traditional GRPO. Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by an average of $\sim$30%, and up to $\sim$70%, while maintaining performance comparable to the model that relies solely on Long CoT. Furthermore, not only does it improve inference efficiency through reduced token generation, but it also brings a $\sim$2$\times$ speedup in training. In addition to the default Adaptive Mode, ARM supports two additional reasoning modes: 1) Instruction-Guided Mode, which allows users to explicitly specify the reasoning format via special tokens—ideal when the appropriate format is known for a batch of tasks. 2) Consensus-Guided Mode, which aggregates the outputs of the three efficient formats and resorts to Long CoT in case of disagreement, prioritizing performance with higher token usage. All the resources will be released.


Peer reviews:
comments: SUMMARY:
The authors propose an Adaptive Reasoning Model (ARM) that can modify how much reasoning is required based on the task difficulty. This ensures performance remains satisfactory without using unnecessary tokens by overthinking. It is trained in two stages: (1) Supervised Fine-tuning to learn reasoning formats, and (2) Reinforcement learning is then carried out to determine the ideal format selection given a task to avoid overthinking.

STRENGTHS AND WEAKNESSES:
Strengths:

+ The paper is flawlessly written and a strong submission. Unfortunately, or not so unfortunately, I cannot articulate many comments as I have very little to critique. The authors motivate their research question extremely well, provide sufficient background, and refer to ample related work to argue the limitations of existing literature. Their work is beautifully presented, and the science is rigorous. Many metrics of interest are investigated and included here in the paper's results. Comparisons between different models are included as well. 

Weaknesses: 

- If I must provide a weakness, it is that the paper avoids low-level technical implementation details, but the methodology that the authors propose is explained very well.

QUESTIONS:
I have no questions at this time.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a framework to train an adaptive reasoning model. Specifically, the framework consists of two stages: (1) SFT to make the model familiar with four reasoning/answering modes. (2) Adaptive GRPO to reshape the reward for avoiding mode collapse when answering. The authors also conducted experiments across multiple reasoning benchmarks to demonstrate the effectiveness of their method in terms of token cost and accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n- The method is reasonable and seems new to me (but I am not an expert in this field).\n- The experiments show significant improvement over the baseline (GRPO).\n- The paper is well-written and easy to follow.\n\n**Weakness**\n- (Major) The authors claimed in the related works that there exist other methods for efficient inference, which are based on \"specialized and length-constrained model training\". The authors also claimed that these estimations are \"not always accurate\". But I didn't see experimental support for this claim. An empirical comparison with these methods will make this work stronger.\n- (Minor) This method requires manually defining four different reasoning modes (and their format/definition). It is not clear to me if (1) defining more fine-grained reasoning modes could help -> we have to rely on such definitions to achieve better reasoning performance (2) the four modes are enough for more open-ended questions beyond the benchmarks.\n\nQUESTIONS:\nIn my understanding, the reward shaping is based on the frequency of different modes. Is this implicitly conditioned on training data distribution? What if the correct answers of training data are biased to just one mode, and other modes will result in a low reward, will this still cause mode collapse? I'd like to see such an analysis, and if we have a way to mitigate this.\n\nLIMITATIONS:\nSee weakness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Adaptive Reasoning Model (ARM), a framework designed to address the overthinking problem in large reasoning models.. ARM is trained to adaptively select from four distinct reasoning formats—Direct Answer, Short CoT, Code, and Long CoT—based on task difficulty. The training involves a two-stage process, SFT followed by a RL stage. For the RL stage, the authors propose Ada-GRPO, which incorporates a format diversity reward to prevent \"format collapse\" to a single, high-accuracy format. Experiments show that ARM achieves comparable accuracy to models using only Long CoT while reducing token usage by an average of 30% and speeding up training by approximately 2x.\n\nSTRENGTHS AND WEAKNESSES:\nGenerally, I think this is a good work tackling the overthinking problem.\nStrengths\n- The experiments are sufficient to support the main claims, especially the additional experiments in Appendix\n- The observation that ARM models learned to select reasoning formats based on task difficulty is convincing\n- The flexibility of ARM, both enabling autonomous pattern selection and explicit guidance is a preferable feature.\n\nWeakness\n- The reported delta in Table 1 is maj@8, and the performance drop in pass@1 would be greater than that. This means ARM reduces generation length at the cost of performance degration.\n- Baselines are not sufficient. Related works such as L1, ThinkPrune should be included in the main experiments for fair comparision.\n- Experiments are only conducted on Qwen models. While recent works figure out potential issues of Qwen, it would be better to try more models like LLaMA and Mistral.\n\nQUESTIONS:\nQuestions\n- How do you organize the SFT dataset? Does every question has 4 different types of answers?\n- Is pass@1 actually avg@8 for multiple samples? This would effectively reduce randomness.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Adaptive Reasoning Model (ARM) to reduce unnecessary long reasoning in LLMs by selecting more efficient formats without sacrificing accuracy. ARM supports four reasoning styles, Direct Answer, Short CoT, Code, and Long CoT, and uses a two-stage training process: supervised fine-tuning on multi-format data, followed by reinforcement learning with Ada-GRPO, a method that encourages diverse format use and prevents overuse of Long CoT. At inference, ARM can adaptively choose formats, follow user instructions, or use a consensus strategy. Experiments across multiple tasks and model sizes show that ARM achieves similar accuracy to baselines while cutting token usage by ~30% and halving RL training time.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- Clarity: The paper is clearly written, with well-illustrated figures, concrete examples, and mostly reproducible implementation details.\n- Empirical Results: Strong performance across a variety of reasoning benchmarks and model sizes, with significant token savings and faster training.\n- Practical Impact: The adaptive reasoning mechanism is lightweight to implement and immediately useful for improving RLVR efficiency.\n\n**Weaknesses**\n- Limited Evaluation Scope: Hard tasks are restricted to math (AIME’25); generalization to non-math domains remains untested.\n- Format Set is Fixed: No ablation on removing or adding reasoning formats; it is unclear whether the approach scales or generalizes beyond the current design.\n- Training Cost: Requires curated multi-format annotations and extensive RL rollouts; this may hinder adoption in low-resource settings.\n- Baseline Fairness: Limited comparisons with other efficiency methods; length-penalty baselines are not directly matched to ARM’s RL setup.\n- Minor Clarity Gap: Some training hyperparameters (e.g., decay schedule) are only described in the appendix.\n\nQUESTIONS:\n**1. Limited task diversity and generalization concerns**\n\n- The evaluation and training setup are heavily concentrated on mathematical reasoning. All reinforcement learning datasets (GSM8K, MATH) and the supervised fine-tuning dataset (AQuA-Rat) focus on math problems, with only CSQA providing limited non-math signals.\n\n- The evaluation includes only one hard benchmark (AIME’25), which also belongs to the math domain. Other difficult tasks, such as complex reasoning (e.g., GPQA, coding), are not considered. This raises concerns about the generalizability of ARM's adaptive behavior. \n\nCould the authors evaluate ARM's performance on a broader range of challenging, non-mathematical tasks?\n\n**2. Training and inference efficiency trade-offs**\n\n- The ARM framework introduces a non-trivial training and data preparation burden. Stage 1 SFT requires each question to be annotated with four distinct rationales and validated for correctness, while Stage 2 RL involves about 1.4 M rollouts per model. This pipeline is more complex than alternatives like length-penalty fine-tuning.\n\n- At inference time, while Adaptive Mode reduces token usage overall, the model still generates Long CoT for 10–20% of easy tasks (Fig. 2), indicating residual inefficiency. Additionally, Consensus-Guided Mode entails multiple generation passes (three efficient formats plus a potential fallback to Long CoT), which can substantially increase cost in high-reliability scenarios.\n\nCould the authors consider potential simplifications of the training pipeline and quantify inference-time overhead under different usage settings?\n\n**3. Lack of ablation on format set**\n- ARM’s design relies on a fixed set of four reasoning formats, but the paper does not justify why these four were chosen or whether they are sufficient. Alternatives like diagram, or tool-calling scratch-pad, are not explored.\n\n- No ablation tests whether removing or adding formats affects performance, token efficiency, or Ada-GRPO’s robustness against format collapse. While Section 4.4 compares formats during inference, it does not analyze the impact of modifying the format set during training.\n\nAn ablation study varying the training format set would strengthen the paper’s claims.\n\n**4. Potential over-fitting**\n\n- The RL stage uses CSQA, GSM8K and MATH, which also appear in evaluation. Even though train/test splits differ, these benchmarks are small and easily memorized.\n\nHow does ARM perform on completely unseen easy/medium/hard benchmarks (e.g. StrategyQA)?\n\n**5. Need for additional experiments** \n\n- Ada-GRPO introduces new hyper-parameters, such as the format-diversity scaling factor, but the authors have not assessed the sensitivity. \n\n- Comparisons with baselines need to be more rigorous: GRPO is evaluated without token constraints, while length-penalty methods (e.g., L1, THINKPRUNE) are tested only on easy tasks. A stronger comparison would include GRPO combined with length penalties and report full token usage under maj@8 or Consensus Mode, rather than per-sample averages. \n\n- ARM claims a 2× training speed-up, no absolute compute cost (e.g., wall-clock hours) is reported.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}