Beyond the Trade-off: Self-Supervised Reinforcement Learning for
  Reasoning Models' Instruction Following
Authors: name: Qingyu Ren, name: Qianyu He, name: Bowei Zhang, name: Jie Zeng, name: Jiaqing Liang, name: Yanghua Xiao, name: Weikang Zhou, name: Zeye Sun, name: Fei Yu

Reasoning models excel in complex problem solving but exhibit a concerning
trade off between reasoning capabilities and instruction following abilities.
Existing approaches for improving instruction following rely on stronger
external models, creating methodological bottlenecks and practical limitations
including increased costs and accessibility constraints. We propose a
self-supervised RL framework that leverages reasoning models' own internal
signals to improve instruction following capabilities without external
supervision. Extensive experiments demonstrate that our framework significantly
improves instruction following capabilities while maintaining reasoning
performance, offering a scalable and cost-effective approach to enhance
instruction following in reasoning models. The data and code are publicly
available at https://github.com/Rainier-rq/verl-if.