Stop Overthinking: A Survey on Efficient Reasoning for Large Language
  Models
Authors: name: Yang Sui; name: Yu-Neng Chuang; name: Guanchu Wang; name: Jiamu Zhang; name: Tianyi Zhang; name: Jiayi Yuan; name: Hongyi Liu; name: Andrew Wen; name: Shaochen; name: Zhong; name: Hanjie Chen; name: Xia Hu
Published: 2025-03-20 17:59:38

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in
complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as
OpenAI o1 and DeepSeek-R1, have further improved performance in System-2
reasoning domains like mathematics and programming by harnessing supervised
fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance the
Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences
improve performance, they also introduce significant computational overhead due
to verbose and redundant outputs, known as the "overthinking phenomenon". In
this paper, we provide the first structured survey to systematically
investigate and explore the current progress toward achieving efficient
reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we
categorize existing works into several key directions: (1) model-based
efficient reasoning, which considers optimizing full-length reasoning models
into more concise reasoning models or directly training efficient reasoning
models; (2) reasoning output-based efficient reasoning, which aims to
dynamically reduce reasoning steps and length during inference; (3) input
prompts-based efficient reasoning, which seeks to enhance reasoning efficiency
based on input prompt properties such as difficulty or length control.
Additionally, we introduce the use of efficient data for training reasoning
models, explore the reasoning capabilities of small language models, and
discuss evaluation methods and benchmarking.