TL;DR: Too Long, Do Re-weighting for Effcient LLM Reasoning Compression
Authors: name: Zhong-Zhi Li; name: Xiao Liang; name: Zihao Tang; name: Lei Ji; name: Peijie Wang; name: Haotian Xu; name: Xing W; name: Haizhen Huang; name: Weiwei Deng; name: Ying Nian Wu; name: Yeyun Gong; name: Zhijiang Guo; name: Xiao Liu; name: Fei Yin; name: Cheng-Lin Liu
Published: 2025-06-03 09:23:41

Abstract: Large Language Models (LLMs) have recently achieved remarkable progress by
leveraging Reinforcement Learning and extended Chain-of-Thought (CoT)
techniques. However, the challenge of performing efficient language
reasoning--especially during inference with extremely long outputs--has drawn
increasing attention from the research community. In this work, we propose a
dynamic ratio-based training pipeline that does not rely on sophisticated data
annotations or interpolation between multiple models. We continuously balance
the weights between the model's System-1 and System-2 data to eliminate
redundant reasoning processes while preserving the model's reasoning
capability. We validate our approach across models on DeepSeek-R1-Distill-7B
and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying
difficulty levels. Our method significantly reduces the number of output tokens
by nearly 40% while maintaining the accuracy of the reasoning. Our code and
data will be available soon.