PEAR: Phase Entropy Aware Reward for Efficient Reasoning
Authors: name: Chen Huang; name: Wei Lu; name: Wenxuan Zhang
Published: 2025-10-09 10:04:31

Abstract: Large Reasoning Models (LRMs) have achieved impressive performance on complex
reasoning tasks by generating detailed chain-of-thought (CoT) explanations.
However, these responses are often excessively long, containing redundant
reasoning steps that inflate inference cost and reduce usability. Controlling
the length of generated reasoning without sacrificing accuracy remains an open
challenge. Through a systematic empirical analysis, we reveal a consistent
positive correlation between model entropy and response length at different
reasoning stages across diverse LRMs: the thinking phase exhibits higher
entropy, reflecting exploratory behavior of longer responses, while the final
answer phase shows lower entropy, indicating a more deterministic solution.
This observation suggests that entropy at different reasoning stages can serve
as a control knob for balancing conciseness and performance. Based on this
insight, this paper introduces Phase Entropy Aware Reward (PEAR), a reward
mechanism that incorporating phase-dependent entropy into the reward design.
Instead of treating all tokens uniformly, PEAR penalize excessive entropy
during the thinking phase and allowing moderate exploration at the final answer
phase, which encourages models to generate concise reasoning traces that retain
sufficient flexibility to solve the task correctly. This enables adaptive
control of response length without relying on explicit length targets or rigid
truncation rules. Extensive experiments across four benchmarks demonstrate that
PEAR consistently reduces response length while sustaining competitive accuracy
across model scales. In addition, PEAR demonstrates strong out-of-distribution
(OOD) robustness beyond the training distribution. Our code is available at:
https://github.com/iNLP-Lab/PEAR.