Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
Authors: name: Xingyu Chen; name: Jiahao Xu; name: Tian Liang; name: Zhiwei He; name: Jianhui Pang; name: Dian Yu; name: Linfeng Song; name: Qiuzhi Liu; name: Mengfei Zhou; name: Zhuosheng Zhang; name: Rui Wang; name: Zhaopeng Tu; name: Haitao Mi; name: Dong Yu
Published: 2024-12-30 18:55:12

Abstract: The remarkable performance of models like the OpenAI o1 can be attributed to
their ability to emulate human-like long-time thinking during inference. These
models employ extended chain-of-thought (CoT) processes, exploring multiple
strategies to enhance problem-solving capabilities. However, a critical
question remains: How to intelligently and efficiently scale computational
resources during testing. This paper presents the first comprehensive study on
the prevalent issue of overthinking in these models, where excessive
computational resources are allocated for simple problems with minimal benefit.
We introduce novel efficiency metrics from both outcome and process
perspectives to evaluate the rational use of computational resources by o1-like
models. Using a self-training paradigm, we propose strategies to mitigate
overthinking, streamlining reasoning processes without compromising accuracy.
Experimental results show that our approach successfully reduces computational
overhead while preserving model performance across a range of testsets with
varying difficulty levels, such as GSM8K, MATH500, GPQA, and AIME.