First Try Matters: Revisiting the Role of Reflection in Reasoning Models
Authors: name: Liwei Kang, name: Yue Deng, name: Yao Xiao, name: Zhanfeng Mo, name: Wee Sun Lee, name: Lidong Bing

Large language models have recently demonstrated significant gains in
reasoning ability, often attributed to their capacity to generate longer chains
of thought and engage in reflective reasoning. However, the contribution of
reflections to performance improvement remains unclear. In this paper, we
systematically analyze the rollouts of eight reasoning models on five
mathematical datasets. We focus on reflective behaviours where the model has
already produced an answer but continues reflecting before finalizing its
output. Our analysis reveals that reflections are predominantly confirmatory
and rarely alter the model's initial answer, a pattern consistent across models
and datasets. To understand the role of reflections in training, we construct
supervised fine-tuning (SFT) datasets with varying amounts of reflection steps.
We observe that training models on rollouts with more reflection steps
primarily enhances first-answer correctness rather than the ability to correct
initially wrong answers through reflections. This motivates us to propose a
question-aware early-stopping method that enhances inference-time token
efficiency by stopping the reasoning process once a few plausible candidate
answers are generated, thereby reducing unnecessary reflection steps. Motivated
by this, we further propose to dynamically truncate the reflections after a
candidate answer has appeared during generation, which reduces reasoning tokens
by 24.5% across five mathematical datasets, within a 2.9% drop in accuracy.