Limits of Generalization in RLVR: Two Case Studies in Mathematical
  Reasoning
Authors: name: Md Tanvirul Alam; name: Nidhi Rastogi
Published: 2025-10-30 23:16:02

Abstract: Mathematical reasoning is a central challenge for large language models
(LLMs), requiring not only correct answers but also faithful reasoning
processes. Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as
a promising approach for enhancing such capabilities; however, its ability to
foster genuine reasoning remains unclear. We investigate RLVR on two
combinatorial problems with fully verifiable solutions: Activity
Scheduling and the Longest Increasing Subsequence, using carefully
curated datasets with unique optima. Across multiple reward designs, we find
that RLVR improves evaluation metrics but often by reinforcing superficial
heuristics rather than acquiring new reasoning strategies. These findings
highlight the limits of RLVR generalization, emphasizing the importance of
benchmarks that disentangle genuine mathematical reasoning from shortcut
exploitation and provide faithful measures of progress. Code available at
https://github.com/xashru/rlvr-seq-generalization.