Title: Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Reasoning

LLM

Long-CoT

Efficiency

R1
Abstract: Recently, long-thought reasoning models achieve strong performance on complex reasoning tasks, but often incur substantial inference overhead, making efficiency a critical concern. Our empirical analysis reveals that the benefit of using Long-CoT varies across problems: while some problems require elaborate reasoning, others show no improvement—or even degraded accuracy. This motivates adaptive reasoning strategies that tailor reasoning depth to the input. However, prior work primarily reduces redundancy within long reasoning paths, limiting exploration of more efficient strategies beyond the Long-CoT paradigm. To address this, we propose a novel two-stage framework for adaptive and efficient reasoning. First, we construct a hybrid reasoning model by merging long and short CoT models to enable diverse reasoning styles. Second, we apply bi-level preference training to guide the model to select suitable reasoning styles (group-level), and prefer concise and correct reasoning within each style group (instance-level). Experiments demonstrate that our method significantly reduces inference costs compared to other baseline approaches, while maintaining performance. Notably, on five mathematical datasets, the average length of reasoning is reduced by more than 50\%, highlighting the potential of adaptive strategies to optimize reasoning efficiency in large language models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose Ada-R1, a two-stage framework for adaptive and efficient reasoning. First, a hybrid model is created by merging Long-CoT and Short-CoT models to enable diverse reasoning styles. Then, bi-level preference training is applied: group-level preference selects suitable reasoning styles based on input complexity, while instance-level preference encourages concise and correct reasoning within each style.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\nThe two-stage design—merging Long-CoT and Short-CoT models followed by bi-level preference training (group-level style selection and instance-level compression)—is both theoretically sound and empirically validated. \nThe authors ensure reproducibility by detailing training procedures, hyperparameters, and dataset compositions. They also plan to release model weights, fostering further research. \nAdaptive reasoning analysis showing that Ada-R1 appropriately increases Long-CoT usage for harder problems (Level 5 in MATH) while favoring Short-CoT for simpler tasks, aligning with theoretical expectations. The contribution is clear.\n\nWeaknesses:\nWhile the experiments focus on mathematical reasoning (GSM8K, MATH, etc.), the paper does not address how Ada-R1 performs on other task types (e.g., logical reasoning, coding, or multi-step decision-making). \nThe paper’s reliance on linear merging might overlook more sophisticated interactions between the two reasoning styles.\nThe paper does not report error bars, confidence intervals, or statistical tests (e.g., t-tests) to validate the significance of performance differences between Ada-R1 and baselines.\nThe Short-CoT models are fine-tuned from Long-CoT models using a small dataset (2,000 samples), which might not fully capture the diversity of short reasoning patterns. This setup could introduce bias, as the Short-CoT models are not independently trained but derived from their Long-CoT counterparts. Comparing against standalone Short-CoT models (trained from scratch) would strengthen the validity of the hybrid approach.\n\nQUESTIONS:\n1. The current experiments focus exclusively on mathematical reasoning tasks (e.g., GSM8K, MATH). How do the authors expect Ada-R1 to perform on other domains (e.g., natural language inference, coding)? Are there plans to validate the framework on non-mathematical tasks? Particularly, i'd like to see the performance on logical reasoning benchmarks such as LogiQA.\n2. The paper reports performance differences (e.g., length reduction, accuracy changes) but does not include error bars, confidence intervals, or statistical tests (e.g., t-tests). Are the observed improvements statistically significant, or could they be due to random variation?\n3.  The paper focuses on reasoning length but does not address the latency or computational cost of dynamically selecting between Long-CoT and Short-CoT during inference. How does the adaptive selection process impact real-time performance?\n4. Recommended citation: Break the chain: Large language models can be shortcut reasoners\n\nLIMITATIONS:\nClearly state in the main text the limitations of the current approach, such as generalization to non-mathematical domains, the choice of linear merging, and the reliance on synthetic Short-CoT models.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Ada-R1, a hybrid reasoning model that combines long and short CoT models to enable diverse reasoning strategies. Ada-R1 is trained using a bi-level preference training approach: at the group level, the model learns to select appropriate reasoning styles, while at the instance level, it is trained to prefer concise and correct reasoning. Experimental results demonstrate both the effectiveness and efficiency of Ada-R1, achieving strong performance with reduced token usage.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The research problem is highly relevant, especially in the context of optimizing test-time computation.\n2. The results are impressive, showing substantial token cost reductions while maintaining performance.\n3. The paper is clearly written and easy to follow.\n\n**Weaknesses**\n1. More out-of-distribution (OOD) and simple reasoning datasets should be included to convincingly demonstrate the model's adaptability. Currently, only two OOD testbeds are used, and the overthinking problem is not exclusive to math. Including a wider range of tasks would strengthen the evaluation.\n2. The definition of group-level preference raises concerns. Since preferences are derived from samples generated by different models—each with distinct problem-solving capabilities—it's questionable whether these preferences can effectively guide the training of a combined model in Stage 1.\n3. The overall training pipeline is quite complex. It involves: 1) training separate models to develop distinct reasoning strategies, 2) merging these models, 3) performing group-level preference training, and 4) conducting instance-level preference training. This multi-step process may limit scalability and practical deployment.\n\nQUESTIONS:\n1. Could the group-level preference stage be removed, keeping only instance-level preference training, since the latter already accounts for reasoning efficiency during data construction?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Ada-R1, a two-stage framework for adaptive and efficient reasoning in LRMs.\nThe first stage constructs a hybrid reasoning model by merging Long-CoT and Short-CoT models, enabling diverse reasoning styles. \nThe second stage introduces bi-level preference training based on DPO, guiding the model to select appropriate reasoning styles at a group level and encouraging concise, correct reasoning at an instance level.\nExperiments on mathematical datasets demonstrate that Ada-R1 reduces inference costs and reasoning length while maintaining performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The core idea of enabling LLMs to adaptively choose reasoning depth (Long-CoT vs. Short-CoT) based on problem complexity addresses a critical efficiency concern in current LRMs.\n* The proposed bi-level optimization (group-level and instance-level preference) is a well-structured approach to guide the model towards both appropriate reasoning styles and conciseness within those styles.\n* The results show large reductions in reasoning length without substantial accuracy loss.\n\nWeaknesses:\n* The methodology for creating the Short-CoT model, specifically fine-tuning a Long-CoT model (DeepSeek-R1-Distill-Qwen-7B) with 2,000 Short-CoT samples from Qwen2.5-Math-7B-Instruct, raises concerns. This approach is uncommon, and the quality and representativeness of these 2,000 samples could significantly influence the fine-tuned Short-CoT model's performance (in Table 2 main results) and the entire analysis in Section 3.2. A more standard or thoroughly justified method for obtaining a consistent Short-CoT baseline would strengthen the experiments.\n* In Section 3.3, the concept of \"CoT Optimization Scope\" as either \"Limited\" or \"Broad\" and its direct implications on model output or final results are not clearly defined or intuitively explained. A more precise definition and illustrative examples of how different scopes manifest in model behavior and outcomes would greatly enhance the understanding of the motivation.\n* The paper lacks statistical significance tests for its experimental results, particularly concerning datasets like AIME25, which has a small test set size (30 samples). The sensitivity of results to hyperparameter settings necessitates multiple evaluation runs with reported averages and error bars to confirm the robustness and generalizability of the findings.\n* Based on the ablation study in Table 3, the \"Merge + group level\" configuration already achieves a significant length reduction with a relatively small accuracy degradation. The incremental benefit of the instance-level preference training appears less pronounced compared to the group-level. The authors should provide a more compelling argument or further analysis to justify the necessity and significant impact of the instance-level preference beyond the group-level.\n\nQUESTIONS:\n* Why is model merging chosen for Stage I over directly training a single model on a mixed dataset of Long-CoT and Short-CoT examples? Please provide a comparative experiment or a more detailed theoretical argument to justify the choice of model merging, demonstrating its advantages over mixed training strategies for creating a hybrid reasoning model.\n* Could the authors elaborate on the precise definition of \"CoT Optimization Scope\" (Limited vs. Broad) and provide more intuitive examples of how these scopes directly influence the model's output and the final task performance?\n* How is the preference margin threshold for the group-level preference determined?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
The paper proposes Ada-R1: a two-stage LM training framework for adaptive reasoning that can potentially limit the length of the reasoning chain depending on the problem. For this, the authors construct a hybrid reasoning model by merging long and short CoT models, then use preference training to guide the model to select suitable reasoning styles. They show that the method reduces the average length of reasoning by ~50%, with minimal degradation to final performance.

STRENGTHS AND WEAKNESSES:
Strengths:
- The proposed two stage training approach, Ada-R1 is novel.
- Ada-R1reduces the average reasoning length by ~50% (on five mathematical datasets,), with minimal degradation to final performance

Weaknesses:
- The two stage approach is more complicated than a baseline e.g. O1-pruner while incurring a limited degradation in performance.
- There is no clear motivation on why Ada-R1 is a better approach than "self training for concise reasoning [17]" or O1-pruner.
- The authors mention in Section 3.3 that other approaches like Overthinking or O1-Pruner "typically operate within a limited optimization scope", whereas their approach is "broad" without explaining what it means or why.

QUESTIONS:
1. Did the authors perform any analysis of samples from AdaR1 vs DPO vs O1-pruner etc to learn what is the difference in reasoning chain? what exactly AdaR1 is doing differently?
2. why upsample MATH dataset for training?
3. Section 6.1, line 254: "This detection method is more generalizable than relying solely on response length." , why?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03