DRIVE: Data Curation Best Practices for Reinforcement Learning with
  Verifiable Reward in Competitive Code Generation
Authors: name: Speed Zhu; name: Jianwei Cai; name: Guang Chen; name: Lulu Wu; name: Saiyong Yang; name: Wiggin Zhou
Published: 2025-11-09 10:11:28

Abstract: Recent reasoning-first models (e.g., OpenAI o1, DeepSeek R1) have spurred a
resurgence of interest in RLVR. Nevertheless, advances are dominated by
mathematics (e.g., AIME), with competitive-programming code generation
underexplored and data curation receiving less attention than RL algorithm
design. We investigate how to construct RLVR datasets (i.e., RL prompts) and
present practical training techniques that yield strong performance on
competitive-programming code generation. Our pipeline begins with supervised
fine-tuning (SFT) distilled from strong open-source models, augmented with
general-purpose and reasoning-intensive data. RL then follows a two-stage
process with executable, testcase-driven rewards: first, training on a large,
uniformly distributed set of competitive-programming problems using Group
Relative Policy Optimization (GRPO) with 8 rollouts per prompt and a relatively
short response-generation window (e.g., 32k during SFT and 24k in this stage)
to expand entropy and mitigate repetition and truncation; second, we perform
Pre-GRPO: updating on a small, high-quality set of challenging
problems with a large rollout budget (64 rollouts per prompt) under a
hard-focus curriculum that continuously retains the most difficult instances
throughout training. We implement our method on Qwen2.5-32B and evaluate on
LeetCode and Codeforces weekly contests to avoid data leakage. The resulting
model achieves state-of-the-art performance among models of similar scale and
is comparable to leading systems such as DeepSeek v3.1 and Doubao-1.5-Thinking.
We also examine scaling trends and observe strong RL scaling on an internal
large-scale MoE model. Our study distills concise best practices for data
curation, entropy expansion, and curriculum design in RLVR for
competitive-programming code generation.