ARM: Adaptive Reasoning Model
Authors: name: Siye Wu; name: Jian Xie; name: Yikai Zhang; name: Aili Chen; name: Kai Zhang; name: Yu Su; name: Yanghua Xiao
Published: 2025-05-26 17:38:50

Abstract: While large reasoning models demonstrate strong performance on complex tasks,
they lack the ability to adjust reasoning token usage based on task difficulty.
This often leads to the "overthinking" problem -- excessive and unnecessary
reasoning -- which, although potentially mitigated by human intervention to
control the token budget, still fundamentally contradicts the goal of achieving
fully autonomous AI. In this work, we propose Adaptive Reasoning Model (ARM), a
reasoning model capable of adaptively selecting appropriate reasoning formats
based on the task at hand. These formats include three efficient ones -- Direct
Answer, Short CoT, and Code -- as well as a more elaborate format, Long CoT. To
train ARM, we introduce Ada-GRPO, an adaptation of Group Relative Policy
Optimization (GRPO), which addresses the format collapse issue in traditional
GRPO. Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by
an average of 30%, and up to 70%, while maintaining performance comparable to
the model that relies solely on Long CoT. Furthermore, not only does it improve
inference efficiency through reduced token generation, but it also brings a 2x
speedup in training. In addition to the default Adaptive Mode, ARM supports two
additional reasoning modes: 1) Instruction-Guided Mode, which allows users to
explicitly specify the reasoning format via special tokens -- ideal when the
appropriate format is known for a batch of tasks. 2) Consensus-Guided Mode,
which aggregates the outputs of the three efficient formats and resorts to Long
CoT in case of disagreement, prioritizing performance with higher token usage.