AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via
  Reinforcement Learning
Authors: name: Chenwei Lou, name: Zewei Sun, name: Xinnian Liang, name: Meng Qu, name: Wei Shen, name: Wenqi Wang, name: Yuntao Li, name: Qingping Yang, name: Shuangzhi Wu

Large Language Models (LLMs) have demonstrated remarkable capabilities but
often face challenges with tasks requiring sophisticated reasoning. While
Chain-of-Thought (CoT) prompting significantly enhances reasoning, it
indiscriminately generates lengthy reasoning steps for all queries, leading to
substantial computational costs and inefficiency, especially for simpler
inputs. To address this critical issue, we introduce AdaCoT (Adaptive
Chain-of-Thought), a novel framework enabling LLMs to adaptively decide when to
invoke CoT. AdaCoT framed adaptive reasoning as a Pareto optimization problem
that seeks to balance model performance with the costs associated with CoT
invocation (both frequency and computational overhead). We propose a
reinforcement learning (RL) based method, specifically utilizing Proximal
Policy Optimization (PPO), to dynamically control the CoT triggering decision
boundary by adjusting penalty coefficients, thereby allowing the model to
determine CoT necessity based on implicit query complexity. A key technical
contribution is Selective Loss Masking (SLM), designed to counteract decision
boundary collapse during multi-stage RL training, ensuring robust and stable
adaptive triggering. Experimental results demonstrate that AdaCoT successfully
navigates the Pareto frontier, achieving substantial reductions in CoT usage
for queries not requiring elaborate reasoning. For instance, on our production
traffic testset, AdaCoT reduced CoT triggering rates to as low as 3.18\% and
decreased average response tokens by 69.06%, while maintaining high performance
on complex tasks.