Title: Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in Large Reasoning Model
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: reasoning, vision language model

Abstract: Reinforcement learning with verifiable rewards (RLVR) has shown an impressive ability to improve LLMs' mathematical and multimodal reasoning capabilities. However, a potential concern with RLVR is the possible regression of the model's general capabilities when finetuning with RLVR. We first re-evaluate the reasoning gains claimed in the literature and track how commonly used reward signals evolve differently during training. We then assess models' general capabilities, such as perception and faithfulness, and observe that performance degrades during RLVR. Motivated by these findings, we propose a replay strategy with dynamic objective reweighting. Our reweighting mechanism adapts in an online manner using short-horizon signals of convergence and instability, shifting the post-training focus away from saturated objectives and toward underperforming or volatile ones. Our method is end-to-end and readily applicable to existing RLVR pipelines. Extensive experiments on benchmarks based on Qwen2.5-VL-3B and Qwen2.5-VL-7B demonstrate the effectiveness of our method, which we not only preserve general capabilities but also improve reasoning by allowing a more flexible trade-off between in-task rewards.