Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for
  Large Language Models
Authors: name: Rui Wang, name: Hongru Wang, name: Boyang Xue, name: Jianhui Pang, name: Shudong Liu, name: Yi Chen, name: Jiahao Qiu, name: Derek Fai Wong, name: Heng Ji, name: Kam-Fai Wong

Recent advancements in Large Language Models (LLMs) have significantly
enhanced their ability to perform complex reasoning tasks, transitioning from
fast and intuitive thinking (System 1) to slow and deep reasoning (System 2).
While System 2 reasoning improves task accuracy, it often incurs substantial
computational costs due to its slow thinking nature and inefficient or
unnecessary reasoning behaviors. In contrast, System 1 reasoning is
computationally efficient but leads to suboptimal performance. Consequently, it
is critical to balance the trade-off between performance (benefits) and
computational costs (budgets), giving rise to the concept of reasoning economy.
In this survey, we provide a comprehensive analysis of reasoning economy in
both the post-training and test-time inference stages of LLMs, encompassing i)
the cause of reasoning inefficiency, ii) behavior analysis of different
reasoning patterns, and iii) potential solutions to achieve reasoning economy.
By offering actionable insights and highlighting open challenges, we aim to
shed light on strategies for improving the reasoning economy of LLMs, thereby
serving as a valuable resource for advancing research in this evolving area. We
also provide a public repository to continually track developments in this
fast-evolving field.