Title: AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning
Venue: NeurIPS 2025
Accepted: no
Keywords: Adaptive Reasoning

Chain-of-Thought

Large Language Models
Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities but often face challenges with tasks requiring sophisticated reasoning. While Chain-of-Thought (CoT) prompting significantly enhances reasoning, it indiscriminately generates lengthy reasoning steps for all queries, leading to substantial computational costs and inefficiency, especially for simpler inputs. To address this critical issue, we introduce AdaCoT (Adaptive Chain-of-Thought), a novel framework enabling LLMs to adaptively decide when to invoke CoT. AdaCoT framed adaptive reasoning as a Pareto optimization problem that seeks to balance model performance with the costs associated with CoT invocation (both frequency and computational overhead). We propose a reinforcement learning (RL) based method, specifically utilizing Proximal Policy Optimization (PPO), to dynamically control the CoT triggering decision boundary by adjusting penalty coefficients, thereby allowing the model to determine CoT necessity based on implicit query complexity. A key technical contribution is Selective Loss Masking (SLM), designed to counteract decision boundary collapse during multi-stage RL training, ensuring robust and stable adaptive triggering. Experimental results demonstrate that AdaCoT successfully navigates the Pareto frontier, achieving substantial reductions in CoT usage for queries not requiring elaborate reasoning. For instance, on our production traffic testset, AdaCoT reduced CoT triggering rates to as low as 3.18% and decreased average response tokens by 69.06% on APP, while maintaining high performance on complex tasks. This substantial token decrease directly translates to a significant reduction in inference computational load. AdaCoT pioneers adaptive CoT triggering, offering a practical and principled solution for developing more efficient, responsive, and cost-effective LLMs, particularly crucial for interactive and resource-sensitive applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces AdaCoT, a reinforcement learning-based framework that enables large language models (LLMs) to dynamically decide when to use CoT reasoning based on the queries. By formulating adaptive reasoning as a Pareto optimization problem, AdaCoT balances performance and computational costs, significantly reducing unnecessary CoT steps while maintaining accuracy on complex tasks. And experiments show AdaCoT achieves Pareto frontier.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n[1] Firstly, this paper is well written. By framing adaptive reasoning as a multi-objective optimization problem, AdaCoT successfully balances the response accuracy and the deployment costs on different tasks, demonstrating its efficiency in LLM reasoning.\n\n[2] This paper presents rigorous experiments across 15 benchmarks and real-world production data, showing significant computational savings of AdaCoT while maintaining competitive performance compared to Full CoT baselines.\n\n**Weaknesses**\n\n[1] The experiments in this work are only conducted on \"internal models\", which raises concerns about the generalizability and reproducibility of AdaCoT to other open-source LLMs. While the proposed framework is theoretically model-agnostic, its dependency on undisclosed models makes it challenging to assess whether the observed significant gains stem mainly from AdaCoT or model-specific traits.\n\n[2] As acknowledged by the authors, the binary CoT triggering mechanism (on/off) may be suboptimal for queries requiring intermediate levels of reasoning. The authors might could discuss how much the fine-grained control over reasoning depth would influence the Pareto frontier.\n\n[3] The principle-guided CoT labeling relies on an auxiliary internal model's judgement, whose biases or limitations may propagate to AdaCoT's training data. Although it is recognized that automated labeling is more feasible and scalable, the potential biases in training data still remains concerns about its impact on the final model.\n\nQUESTIONS:\n[1] The experiments of AdaCoT framework mostly relies on internal models. Could the authors provide the performance of applying AdaCoT to other open-source models, and analyze the impact of different base models on the framework?\n\n[2] What is the intuition of the design of SLM, or why does excluding the loss of the token immediately succeeding the <think> tag help? Are there any theoretical or empirical explanations?\n\n[3] How does AdaCoT handle ambiguous or adversarial queries, like questions that seem simple but requre implicit reasoning? Are there failure cases where it incorrectly skips CoT, and what is the frequency?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an AdaCoT framework to enable LLMs to adaptively decide when to invoke a long chain-of-thought through reinforcement learning. The paper instructs LLMs to label training queries as either likely benefiting from CoT or likely suitable for a direct answer automatically. Based on the labels, AdaCoT uses SFT and RL to LLMs to assess the complexity of each query internally and decide when to use CoT reasoning. A masking strategy is proposed to avoid the decision boundary collapse when training data is imbalanced. Experiments are conducted on internal 15B/150B MoE LLMs to verify the method's effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n-\tTeaching LLMs to differentiate complex and simple questions is a valid research topic.\n-\tThe writing is clear, and the proposed method is intuitive.\n-\tExperiment results show the effectiveness of AdaCoT, which significantly cuts down CoT rate and token usage without a significant decrease in task performance.\nWeaknesses:\n-\tThe proposed method is only verified on internal models, which affects the reproducibility of the work. The trained LLM is not directly compared with other open-source LLMs regarding task performance.\n-\tIn line 119, queries are labeled as either likely benefiting from CoT or likely suitable for a direct answer. The correctness of the labeling process should be verified, e.g., the overlap ratio of the queries labeled as needing CoT by human annotators and LLMs. Besides, being labeled as needing CoT by both humans and LLMs doesn’t necessarily mean the query indeed needs CoT.\n-\tAdaCoT uses the </think> token to signal whether to use CoT. The LLMs learn to predict </think> after <think> directly if no CoT is needed. Thus, a direct baseline that should be compared is to use a simple classification head on the representation of the <think> token from different LLM layers.\n\nQUESTIONS:\n-\tThe CoT usage is measured by the CoT triggering rate. Could other metrics, e.g., the tokens used in the response, be an alternative measurement? Because the LLMs can also generate long chain-of-thought outside the <think></think> tokens.\n-\tCould you please conduct an additional experiment that uses ground truth ‘needed CoT’ labels derived from LLMs to instruct LLMs to respond to open-source benchmarks? The reported performance can be used to verify the correctness of the automatic labeling process.\n-\tCould you please provide the comparison with a naïve method that uses a simple classification head to determine whether the CoT is needed or not?\n-\tIn the web applications of ChatGPT and DeepSeek, there is a button that lets users decide whether to enable long chain-of-thought or not. What’s the advantage of allowing LLMs to decide by themselves? Is it possible for the LLMs to make better decisions than humans?\n\nLIMITATIONS:\nNo. The methods should be verified on open-sourced LLMs to confirm the universality. The quality of the automatic labeling process should be further tested.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper introduces AdaCoT, an RL approach that frames the “generate CoT or not” decision as a Pareto trade-off between accuracy and inference cost. A PPO policy—penalized for missed or redundant reasoning and stabilized with selective-loss masking—learns when to emit CoT. On 15 benchmarks, AdaCoT matches always-CoT accuracy while cutting reasoning traces by 30–60%. In production, it lowers CoT calls to 3–13% and trims token usage by ~69%, without sacrificing complex-task performance.

STRENGTHS AND WEAKNESSES:
**Strengths**

1. Propose a principled formulation to cast adaptive reasoning as explicit multi-objective (Pareto) optimisation.

2. Experiments show substantial real-world savings: up to 69 % fewer tokens and over 95 % fewer CoT calls.

**Weaknesses**

1. The binary on/off trigger ignores variation in CoT length — a pivotal factor for truly adaptive reasoning.

2. The initial “CoT-needed” labels are generated by another LLM, so mislabeled seeds might bias the policy.

QUESTIONS:
1. Could multi-level CoT length (not just on/off) be learned under the same Pareto framework?

2. How well does the trigger policy transfer to a smaller or multimodal base model without re-training?

3. What happens when the reward model is noisy or domain-shifted—does SLM still prevent collapse?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes AdaCoT, a novel method that enables LLMs to adaptively trigger Chain-of-Thought reasoning based on query complexity. AdaCoT optimizes model performance with reduced computational costs using Pareto-optimal reinforcement learning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well-structured and easy to follow.\n\n- The idea of adaptively triggering cot to achieve better performance while controlling computation cost is novel.\n\n- The empirical results in experiments are strong evidence for the effectiveness of AdaCoT.\n\nWeaknesses: see Questions.\n\nQUESTIONS:\n- Lack of clarity on training datasets and experimental details. The paper mentions two RL stages, RL-Math and RL-General, but provides no specific details on the datasets or training procedures used in each stage. Clarifying the exact data sources, domain coverage, and training configurations would help readers better understand how the model is trained and the scope of the evaluation.\n\n- The paper does not include a comparison with relevant prior works, such as Arora et al. [1], which also focuses on reduce omputational costs while maintaining high performance. Including such comparisons would provide a clearer perspective on how AdaCoT stands in relation to existing methods for optimizing reasoning efficiency in language models.\n\n- The paper claims to focus on adaptive reasoning, distinguishing itself from previous works that primarily focus on reducing compute costs. However, there is no empirical evidence provided to demonstrate how adaptive reasoning in AdaCoT leads to better user experience or performance. A deeper analysis comparing the impact of adaptive reasoning on real-world tasks would strengthen the claims. \n\n- Incorporating a human evaluation could provide valuable insights into how AdaCoT improves reasoning efficiency and user interaction, helping to validate the practical benefits of adaptive CoT triggering.\n\n\n\n\n[1] Arora, Zanette, Training Language Models to Reason Efficiently\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}