Scheduling Your LLM Reinforcement Learning with Reasoning Trees
Authors: Hong Wang
Zhezheng Hao
Jian Luo
Chenxing Wei
Yao Shu
Lei Liu
Cheaterlin
Hande Dong
Jiawei Chen
Venue: ICLR 2026 Poster (2026)
Keywords: large language model
RLVR
Data Scheduling
Primary area: foundation or frontier models, including LLMs

Abstract: Using Reinforcement Learning with Verifiable Rewards (RLVR) to optimize Large Language Models (LLMs) can be conceptualized as progressively editing a query's 'Reasoning Tree'. This process involves exploring nodes (tokens) and dynamically modifying the model's policy at each node. When combined with data scheduling, this process yields further gains in data efficiency and accuracy.
However, existing RLVR data scheduling methods typically rely on path-based metrics to rank queries, overlooking the reasoning tree structures of these queries.
In this paper, we introduce a novel metric, namely **Reasoning Score (r-score)**, which measures the query's learning difficulty based on the structure of its reasoning tree.
Based on the r-score, we propose the **Reasoning Tree Schedule (Re-Schedule)**, a scheduling algorithm that constructs a curriculum progressing from structurally simple (high r-score) to complex (low r-score) queries.
Experiments on six math-reasoning benchmarks show that Re-Schedule significantly improves average accuracy, achieving gains of up to 3.2\%.
These strong results validate our approach and demonstrate that a structural understanding of the reasoning tree provides a more powerful and principled foundation for RLVR data scheduling.

Published as a conference paper at ICLR 2026
SCHEDULING YOUR LLM REINFORCEMENT LEARN- ING WITH REASONING TREES
Hong Wang' Zhezheng H
Jian Luo
Chenxing Wei'
Yao Shu* Lei Liu
Qiang Lin'
Hande Dong' Jiawei Chen
1 Tencent
ZZeeiiang University
Independent Researcher
Hong Kong University of Science and Technology (Guangzhou)
(hwhongwang, [haozhezheng, sleepyhunt 2 ju edu... en
com
ABSTRACT
Using Reinforcement Learning with Verifiable Rewards (RLVR) to optimize Large Language Models (LLMs) can be conceptualized as progressively editing a query's 'Reasoning Tree'. This process involves exploring nodes (tokens) and dynamically modifying the model's policy at each node. When combined with data scheduling. this process yields further gains in data efficiency and accuracy. However, existing RLVR data scheduling methods typically rely on path- based metrics rank queries, overlooking the reasoning tree structures of these queries. In this paper, we introduce a novel metric, namely Reasoning Score (I-score), which measures the query's learning difficulty based on the structure of its rea- soning tree. Based on the r-score, we propose the Reasoning Tree Schedule (Re- Schedule), scheduling algorithm that constructs curriculum progressing from structurally simple (high r-score) to complex (low r-score) queries. Experiments six math-reasoning benchmarks show that Re- Re-Schedule significantly improves average accuracy, achieving gains of up to 3.2%. These strong results validate our approach and demonstrate that structural understanding of the reasoning tree provides more powerful and principled foundation for RLVR data scheduling"
INTRODUCTION
Advancing the complex reasoning capabilities of Large Language Models (LLMs) remains signif- icant challenge, particularly in domains like mathematical problem-solving. Reinforcement Learn- ing with Verifiable Reward (RLVR) (Gao et al., 2024 DeepSeek-AL et al. 2025 J, especially through policy optimization methods like GRPO (Shao et al. 2024). has emerged as a powerful paradigm to address this challenge. As shown in Figure I (a), in this framework, the space of potential solution paths for query can be modeled as a specific 'Reasoning Tree' (Wang et al. 2025ee] Yang et al.) 2025b), where each node represents an intermediate reasoning step and each path represents a poten- tial solution trajectory. From this perspective, RLVR operates dynamic 'node-editing" process of the reasoning tree: by rewarding correct paths and penalizing incorrect ones, the model iteratively refines its decision policy at each tree node. This optimization process gradually prunes branches that lead to low-quality or incorrect solutions, thereby improving overall reasoning accuracy.
In this paradigm, data scheduling plays critical role in model performance (Hu et al., 2025 et al. 2025 Yang et al., 2026 Ren et al.). The concept of data scheduling originates from curricu- lum learning (Bengio al. 2009). which posits that models learn more effectively when training examples (queries) are organized in a meaningful sequence. Existing data scheduling strategies typi- cally pre-define 'difficulty' metric for queries, and and schedule them from easy to hard to improve data efficiency and final performance (Xi et al., 2024 Chen et al. 2025b Dai et al. (2055) How- ever, from a reasoning tree perspective, current difficulty measure strategies exhibits a critical limi- tation: current methods estimate difficulty primarily via final solution accuracy, overlooking richer
"Equal Contribution. "Corresponding Authors
Our code is available http::/gihha..com/cooooooeeeehee

---

Published as a conference paper at ICLR 2026
Query (q1)
Reasoning Tree
Step
Accuracy (%)
(a)
100%
Train
Step
(b)
Query
Query
Train step
Accurncy 25% 50% - 100%
Query (q2)
Reasoning Tree
Serecied Mrde eooe
(c)
Train
Step
Acourncy 50% 62.5% 75% 87.555 1009E
Figure I: (a) A simple reasoning tree (ql) requires less node editing for performance improvement than complex one (q2). (b) Consequently, q1 shows high training efficiency (steep learning curve) despite low initial accuracy, while q2's complex structure leads to low efficiency. (c) Our method leverages this structural insight significantly outperform baselines on various datasets.
query-level characteristics such the structural complexity of the reasoning tree. Accuracy alone insufficient - low accuracy does not necessarily indicate that a query is inherently hard, and high accuracy does not guarantee ease of optimization. This inconsistency can undermine the efficacy of accuracy-basse scheduling approaches. We illustrate this issue with the following examples.
To illustrate, consider two representative queries, q1 and q2, whose reasoning trees shown in Figure a As depicted in Figure [b)) LLMs may exhibit low initial accuracy on ql, due to the presence of many incorrect solution trajectories (reasoning paths). However, its simple tree structure means that modifying a few key decision nodes can yield substantial accuracy gains, indicating high learning efficiency despite the poor initial performance. In contrast, q2 achieves higher initial accuracy, with roughly half of its trajectories being correct, yet these correct paths are scattered across disparate subtrees. This fragmented structure requires more extensive edits across numerous tree nodes, typically resulting in higher training difficulty and lower learning efficiency. Critically, existing path-based metrics will misinterpret q1's 's low accuracy as high difficulty. thus assigning lower training weight, while incorrectly prioritizing the more difficult q2. Such path-based metrics may lead to a less efficient training process. This motivates our central research question: How can we move beyond path-based metrics to directly quantify query's 's true learning difficulty from its reasoning-tree structure?
To address this question, we introduce the Reasoning Score (r-score), novel metric that quanti- fies query's learning potential based on its reasoning tree structure. We formalize this by framing the reinforcement learning training process budget', which we define as a fixed number of node editing operations. Consequently, a query's 's r-score is its maximum potential accuracy gain achievable within this limited editing budget. This metric clearly explains the discrepancy in our example: ql, with its "concentrated" error struc- ture, yields a high r-score because a small budget (e.g., two edits) produces massive accuracy gain (+75%), Conversely, q2's 'diffuse" structure results in a low r-score, as the same budget only yields marginal improvement (+25%). Therefore, higher r-score signifies a more tractable reason- ing structure and greater learning efficiency, offering more comprehensive assessment of difficulty than path- based metrics.
optimization problem under finite 'node editing
Building on the Reasoning Score, we propose the Reasoning Tree Schedule (Re-Schedule). novel data scheduling algorithm designed to guide RLVR training more efficiently. Our method consists of three main stages. First, an offline approximation of each query's reasoning tree is constructed by sampling multiple solution trajectories from a base model. Second, this approximated reasoning tree is used to calculate each query's 's reasoning score by simulating the editing process. Finally, we integrate the r-score as a dynamic weight into the RLVR loss function to form a schedule. This schedule prioritizes high-scoring (simple) queries in the initial training phases to accelerate convergence on simple queries. As training progresses, the weighting gradually shifts to lower- scoring (difficult) queries, enabling the model master more challenging problems.
2

---

Published as a conference paper at ICLR 2026
In summary, the main contributions of this paper are:
We introduce the Reasoning Score (r-score), new tree-based metric that measures a query's s learning efficiency rather than its path-based solution accuracy.
We propose Re-Schedule, a data scheduling algorithm that uses the r-score to create an effective, easy-to-hard curriculum for RLVR.
As shown in Figure O)), we empirically demonstrate that our approach significantly im- proves average accuracy, achieving gains of up 3.2%, complex reasoning tasks.
2 RELATED WORK
2.1
REINFORCEMENT LEARNING WITH VERIFIABLE REWARDS IN LLMS
Reinforcement learning with verifiable reward (RLVR), where the reward computed by rule- based verification function, has been shown to be effective in improving the reasoning capabilliiess
of LLMs Gao et al.) 2024; DeepSeek-At AI et al. 2025 Kimi et al. 2025 Zeng et al. 2025; Wen Ma et al.). Typically, RLVR frameworks
et al. 2025 Song et al. 2025 Wei et al. 2025b a
assign binary reward by comparing the model's 's generated output against a ground-trum
solu-
tion, indicating whether it is correct incorrect. This reward design obviates the need for complex outcome- based or process- based reward models, offering a straightforward yet potent approach. Re- cent advancements in policy optimization algorithms, such PPO and GRPO, have further refined this paradigm (Schulman et al. 2017]] Kazemnejad et al. 2024 Yuan et al. 2025 Yue et al.) 2025
Shao et al.
2024 Yu et al. 22255 Liu 2025 Zhang al., 2025a] Hu [025 Hao et al.) 2025b
Wei et al.
2025c Liu et al. 2026 Yang et al. 2025a). In contrast to these studies, which focus
on algorithmic improvements, our work builds upon the standard GRPO framework with primary
focus on designing more effective training data schedule.
2.2 DATA SCHEDULING ALGORITHM IN LLM REINFORCEMENT LEARNING
Various data scheduling strategies have been proposed to enhance the reasoning capabilities LLM Reinforcement Learning. These can be broadly categorized into static selection and dynamic ad- justment methods. Representative of static selection is LIMR (Li et al., 2025), which selected 1.4k examples from 8.5k set for RLVR to match the performance of using the full set. In contrast, dynamic strategies make real-time adjustments during training. For instance, R² employs reverse curriculum reinforcement learning simplify the model's 's exploration space (Xi et al. (024)) LPPO Chen et al. 2025b) utilize the gradient of accuracy to prioritize data, effectively treating learning difficulty derivative of performance. Similarly, Seed-GRPO (Chen et al. 2025a) employs mantic diversity (uncertainty) as a proxy for difficulty. Furthermore, DELT leverages training gradients measure the quality and learnability of data (Dai al. 2025). subsequently adjusting sample weights. However, these methods rely on outcome-based proxies (e.g., accuracy), effec- tively treating reasoning as a flat sequence (Zhao et al.) 2025 Zhang al., 2025b). They overlook the inherent tree- structruut solution space of reasoning tasks. In contrast, our approach explicitly leverages this topologiaal structure. By analyzing the Reasoning Tree, we directly quantify query's 'structural learnability', providing more precise and principled measure of difficulty than perfor- mance statistics alone.
PRELIMINARIES
3.1
GROUP RELATIVE POLICY OPTIMIZATION
The objective of the GRPO algorithm is to optimize a policy सy based on a group of generated responses (Shao al., 2024 Yu et al. 2025). For query q from dataset DD the policy generates G responses {o T. The token-level objective function is formulated as:
G lod
[(0) = E, E. (aa)1
min (ri,AA,, (lippr,, 1 E. 1.1++00 E)
G
lop|
i=l
3

---

Published as a conference paper at ICLR 2026
where ri,t (.. is the probability ratio of the token oi, between the current and old policies. The advantage term As, is constant for all tokens within single response and calculated by normalizing the response's s reward Rg relative to the other responses in the group:
ALe R; - mean (6m))
11,
(2)
std (Rx) 1k= +0
where 8 a small constant for numerical stability.
Data scheduling algorithms can be formulated by introducing weighting function w(g, t) that mod- ulates the contribution of each query E D and current epoch to the overall objective. Specifically, the objective in Equation iis modified follows:
Scchccule (0) D...,... (w) (original objective term for 4))
Note: In the equations above, we have abbreviated the full objective for clarity. For example, in an accuracy-basee based curriculum learning, the training weight w is formulated as a function of the query's accuracy ACC(q) and current epoch t:
a(ACC()),) 1) (1-2(1))AC(0)) + (1) ACC(4)),
(4)
w rank(a)% Wmax + (1 (aank(a))9)) Wiin-
(5)
Here, Wmax and Wiiin are hyperparameters that define the maximum and minimum training weights (e.g., Wmax 0.8, Wmin 0.2): And rank(a) means calculating the reverse order of in the entire dataset. The term (t) is a scheduling function that progresses over time. Common choices for (() include a linear mapping, >()) = t/T, or a sigmoid function, 7(1) ((+ - ( o(x) =
(1 1le e =- where the total number of epochs.
3.2 REASONING TREE
For complex reasoning tasks, the process generating solution can be conceptualized as traversing a RRasoning Tree'. In this context, the root of the tree is the initial prompt, and each node represents a partial solution or an intermediate reasoning step. The branches extending from node correspond to the possible next tokens or thought segments that the LLM can generate.
Due to the combinatorial explosion of possible solution paths, the complete reasoning tree is typ- ically computationally intractable. Therefore, analysis often relies a structured approximation (e.g., a fixed-structure k-ary reasoning tree). Formally, an approximated reasoning tree is defined as a triplet 7' = EN,,R where N is the set of nodes, E is the set of edges, and R. defines the parent-child relationships.
The components of the tree described using the fol- lowing notation: Nigat C. C is the set of leaf nodes; For a given node n, EN, C(n:) denotes the set of its immedi- ate children and [(n:) denotes the set of its leaf descen- dants. If n4 a leaf node, then L(n+) framework, each non-leaf node n, E N Weat represents partial reasoning path, while a complete path leaf node ny E Niat corresponds to a full solution trajectory.
100-
99.09
$6.55
72.15
{n.). Within this
1S1
Ascunary
6
From this perspective, the RLVR optimization process is a dynamic 'node editing' of this reasoming tree. By re- warding correct paths and penalizing incorrect ones, the policy optimization algorithm adjusts the token probabil- ities at each node, effectively strengthening the branches that lead to correct answers and weakening those that lead to errors. The structure of this tree tree-the distribution of correct and incorrect paths- is intrinsic to each problem sample and, as we will argue, is a key clue to its learning dynamics.
Traising Spochs
Figure 2: Accuracy Progression During Training. The solid line represents the average accuracy, and the shaded area indicates the range.
4

---

Published as a conference paper at ICLR 2026
Reasoning Tree Schedule (Re-Schedule)
query q
dataset D
200 tokens
query
resoning tree T (W, R)
(a)
sample
LLM
ACC
query 9
score max NN Σn R(n:))
R(q)
ACC
maximum potential accuracy
gain achievable within this limited editing budget
(b)
training weight
weight
a((()), t) = (1 - (t)) + rank(o) (a)) wmax + (1 rank(a)) (
R high
R low
-imin
(c)
objective function
sehbedule (0) = Bg. [w(q, t) (original objective term for 4)]
epoch
Figure 3: Overview of the Reasoning Tree Schedule (Re-Schedule) Algorithm. Tree Construc-
tion: For each query, an approximate reasoning tree is constructed by sampling multiple solution paths from base model (Note: This figureis for illustrative purposes only; our experiments use tree with depth of 4 and a width of 4, i.c., k = 4, d 4.). (b) R-Score Calculation: The tree's structure is analyzed to compute the r-score, metric quantifying the query's learning potential. (c) Dynamic Weighting: The r-scores are used to dynamically weight each query during training, forming curriculum that progresses from structurally simple (easy) complex (hard) examples.
4 MOTIVATION
The premise this work is that path-based metrics such as accuracy are poor indicators of query's true learning difficulty. To illustrate our point, supplement the example from the introduction with an experiment. As shown in Figure 2 we selected two distinct sets of 100 queries each from the DAPA-Math-177 dataset, using the Qwen2.5-Math-72 model. The blue line represents 'Stag- nant Samples' -quries with high initial accuracy but complex reasoning structures (low r-score). Their flat learning curve indicates that despite high initial performance, they difficult to improve further. In contrast, the red line represents 'Potential Samples' queries with low initial accuracy but simple tree structures (high r-score). Their steep learning curve demonstrates high learnability, where a small amount of training yields significant gains. This discrepancy highlights that path- based metrics, like accuracy, are biased measurements for learning difficulty. This finding motivates us to design new metric based on the structure of the reasoning tree.
5 METHOD
As illustrated in Figure 3 the Reasoning Tree Schedule (Re-Schedule) enhances reinforcement learning performance by creating curriculum based on our novel metric, the Reasoning Score (r-score). The r-score quantifies query's 's learning difficulty a priori based on the structure of its reasoning tree. Next, we will introduce the specific implementation details.
TREE CONSTRUCTION
As the entire reasoning tree is computationally intractable, construct a manageable, fixed- structure k-ary approximation for each query q. The structure of this tree, T. is defined by a branch- ing factor kt maximum depth d, and token interval / (e.g., 4, d 4,7 200).
5

---

Published as a conference paper at ICLR 2026
The construction process begins the root node (the query 9) and proceeds via a periodic branching strategy during response generation. Specifically, a branch is triggered immediately at the beginning of the response and subsequently at every I-token interval. As shown in Figure (a), at each trig- ger, the current path splits into k independent sub-paths that continue to generate parallel. This recursive branching process continues until predefined maximum depth d is reached. To minimize computational overhead from this mulli--pth sampling, we use the Key-Value (KV) Cache, as all sibling branches share the same prefix.
In RLVR tasks, a solution's quality is determined by the correctness of its final answer, which corresponds to a leaf node in our framework. Therefore, we define the quality of any intermediate node n, as the average accuracy of its leaf descendants, [(n)). This is quantified using an accuracy function:
ACC(S) Enjes [n is correct)
66)
|S]
where is a set of leaf nodes and I[(-) is the indicator function. This allows us to assess quality different levels: the quality of reasoning segment via ACC(C(n:)) and the model's aggregate performance on the query via ACC(/)ear).
5.2 R-SCORE CALCULATION
The r-score quantifies the learning potential of a node or query by measuring the maximum achiev- able accuracy gain under limited policy refining cost, like a limited node editing budget. Given this idea, for any non-leaf node n,, we define its r-score, R(n:). as the maximal accuracy gain achievable by selecting its single best child branch and pruning all others. This is formulated as:
R(n,)
max ACC[NNaat \, [(n,) U [(namu) ACC
new (m(.)
The overall r-score for a query. R(g), estimates the total accuracy gain achievable under a budget that limits modifications to maximum of M nodes. It is the maximum sum of r-scores from any set of M non-conflicting nodes (e.g., for a budget of M = 4):
max R(4) I-
Σ R(n.).
(8)
M.EV.
Two nodes are considered conflicting if one located in a subtree that is implicitly pruned by the optimal branch selection of the other.
Intuitively, solving Equation 7 represents the evaluation process of the sub-tree's structure, while a simpler structure of reasoning tree starting from n, yields higher R(n.). Combining the evalua- tion R((:)) of each node n, under a limited budget M, solving Equation (8) to find the maximum achievable accuracy gain over the reasoning tree, like exploring possible combinations of M nodes and picking the best combination. Thus, a higher R(g)) indicates that substantial accuracy improve- ments can be made by correcting just a few critical reasoning steps, signifying a structurally simple and efficient-to -o-cert query.
5.3 DYNAMIC WEIGHTING
To strike balance between data diversity and data scheduling, propose weighted scheduling framework that dynamically adjusts data prioritization. Specifically, queries are assigned adaptive weights determined by both training step and r-score Rt. Specifically, when it is early training stage, higher weights are assigned to samples with higher r-scores (indicating lower learning dif- ficulty), stabilizing the reinforcement learning. When RL training meets the later training phase, queries' weights will be redistributed gradually towards lower-r-score samples (higher learning dif- ficulty) to enhance model generalization.
Motivated by this, the training weight w of each query is formulated as
al a(R(g), t) (1- (t)) (1- ((g) + 7(t) -R(g),
(9)
(a))( Wmax (1 rank(a))% Wminn
(10)
where is the current epoch; Wmax and Wminn are hyperparameters that define the maximum and min- imum training weights; And rank(a) means calculating the reverse order of in the entire dataset;

---

Published as a conference paper at ICLR 2026
over offferent Aocetixies
Vieeed
Uiteed
Trunig Step
The 22g Sepp
The 1i Sepp
P
[8
Figure 4: (a) The average MCN decreases over time, indicating successful optimization. (b) & (c) To compare metrics, we train models on the top 1/3 of data selected by each. The plots show the resulting (b) training accuracy and (c) test accuracy. The model used is Owen2.5-Math-77B
(ff can be either linear mapping (t) = = sigmoid y(t) = - -.5)). The a(R(g), t) is monotonically varying function that down-weights high-scoring (simple) queries over time while up-weighting lower-scoring (difficult) ones. This scheduling approach balances exploitation of eas- ily learnable patterns and exploration of challenging instances, mitigating catastrophic forgetting of underrepresented data distributions.
ANALYSIS
6.1 TRAINING AS REASONING TREE OPTIMIZATION
To empirically validate that the training process is optimizing reasoning trees, we conducted an experiment centered on new metric: the Minimum Corrective Nodes (MCN). This metric is defined as the minimum number of node modifications required for the reasoning tree to achieve specified target accuracy. A single node modification counted as one token change; thus, a lower MCN signifies a well-structured reasoning tree. In our experiment, we tracked the MCN on the DAPA- Math-17K training set during the training of Swen2.5-Mathh excluding queries where the base model's accuracy was below 10%.
As shown Figure 4 (), the average MCN across the training set exhibits a consistent downward trend training progresses, regardless of the target accuracy. This result demonstrates that the rein- forcement learning process effectively refines the model's policy at critical decision nodes, thereby validating our central assumption that training process of reasoning optimization.
6.2 THE RELATIONSHIP BETWEEN R-SCORE AND LEARNING DIFFICULTY
In this experiment, we want see which metric best identifies valuable queries for early-stage training. The process is as follows: First, we use each metric to select the top one-third of the data, creating several distinct subsets. Second, we train separate model on each of these subsets for single epoch. Finally, we evaluate the resulting models on both the training and test sets.
As shown Figure 4b), the subset selected by the ACC-based method initially shows higher av- erage accuracy on the training set, as expected from its selection criteria. However, as training progresses, the model trained on the r-score-selccted subset quickly surpasses This indicates that the r-score is more effective at identifying queries with low learning difficulty, rather than just initial accuracy.
The advantage of r-score is even more evident on the test set, as shown in Figure c). Here, the model trained on the escore-selected queries consistently outperforms both the ACC-based selection and baseline with random query selection (GRPO). This confirms that the queries identified by the r-score provide the most effective learning signal, leading to better performance improvement and validating its capability in identifying the real difficulty of queries.

---

Published as a conference paper at ICLR 2026
7 EXPERIMENT
7.1 RL TRAINING SETUPS
Training setting We conduct experiments on two different models, including Qwen2.5-Mir -Mtthh--- and Qwen2.5-7B. We adapt our training codebase from verl (Sheng et al. 2025) and follow the training recipe of standard GRPO. Our training data is DAPO-Math-17K (Yu 2025). containing only math problems with integer ground-truth answers. Both the KL.-divergence and entropy loss terms removed in experiments (Hao al.) 2025b). Generation batch size is set to 512. Training is performed with top-p value 1.0 and temperature 1.0.
Evulation We evaluated models and baselines six widely used mathematical reasoning benchmarks: AIME24, AIME25, AMC23 (Li et al. 2024). MATH-500 (Hendrycks et al. 2021), Minerva Math (Lewkowycz et al., (022), and Bench (He et al. 2024). Validation is per- formed with a top-p value of 0.7 and temperature 1.0 across all models and test sets. We use Math-Verify for training, validation, and final evaluation. We report avg @ 32 for all datasets. All results are presented as percentages.
Baselines For the throughout comparison, compare method against 7 baselines, including standard GRPO (Shao et al. (204)), SimpleRL-Zoo (Znn et al. 2025), Eurus-PRIME(Cui et al., 2025). OPO eH al. (025a), ACC (curriculum learning based on accuracy, using sigmoid weight- ing), LPPO (Chen et al. 22026)) and Seed-GRPO (Chen et al. 2025a).
Our Methods Re-Schedule is implemented with two weighting schemes: 'linear' and 'sigmoid', Unless otherwise specified, the reasoning trees in our experiments are constructed with a branching
factor of k: 4, maximum depth of d 4, and token interval of 200. The weighting schemes are defined as follows: 1. The 'linear' scheme uses (t) = t/T; 2. The 'sigmoid' scheme
uses 7() = ( ( 0.5)). For both, we set the total number of epochs T 10. Details of the training setup can be found the Appendix
7.2 MAIN EXPERIMENT
Table I: Main benchmark results Qwen2.5-Math-7B. All values accuracies multiplied by 100. Best results are in bold.
Model
AIME24 AIME25 AMC23 MATH500 Minerva Olympiad Avg.
Qwen2.5-Math-77
13.8
44.6
39.6
13.8
21.2
Classical RLVR Methods
GRPO
28.0
14.3
66.2
78.6
37.5
40.9
44.3
SimpleRL-Zoo
30.8
14.2
65.4
79.2
37.1
40.8
44.6
Eurus-PRIME
20.9
13.0
65.2
79.8
37.5
40.6
42.8
OPO
32.2
13.4
71.5
82.2
38.6
41.0
46.5
Scheduling Methods
ACCsigmoid
31.5
15.6
70.9
80.8
38.6
42.2
46.6
LPPO
32.8
14.9
63.3
79.2
39.0
40.6
45.0
Seed-GRPO
30.7
14.0
71.0
80.0
38.2
38.5
45.4
Our Methods
Re-Schedule
34.2
15.6
72.4
81.2
36.4
42.5
47.1
Re-Schedule, siggmoid
35.2
16.0
72.3
82.2
42.3
44.4
48.5
As shown in Tables and our Re-Schedule method consistently sets a new state-of-the-arr, achiev- ing average scores of 48.5 on Qwen2.5-Math- and 44.5 Qwen2.5-7B. It significantly outper- forms both scheduling baselines like ACCsigmosd (by up to 3.2%) and classical RLVR methods like OPO/GRPO (by up 3.8%). These results validate central claim: that the reasoning tree structure, captured by our r-score, is a more effective way to measure the real learning difficulty of query than path-based metrics like accuracy.
8

---

Published as a conference paper at ICLR 2026
Table Main benchmark results on Qwen2.5-7B. All values are accuracies multiplied by 100. Best results are in bold.
Model
AIME24
AIME25
AMC23 MATH500 Minerva Olympiad Avg.
Qwen2.5-7B
5.1
2.5
27.8
34.4
5.9
13.5
14.9
Classical RLVR Methods
GRPO
15.6
8.8
62.5
78.2
38.6
40.4
40.7
SimpleRL-Zoo
17.0
9.6
64.7
76.6
31.6
40.3
40.0
OPO
16.6
8.4
64.6
74.6
31.6
40.3
39.4
Scheduling Methods
ACC ACC sigmoid
16.7
9.8
68.6
79.0
34.2
39.4
41.3
LPPO
15.8
9.4
64.0
76.8
35.3
36.7
39.7
Seed-GRPO
13.3
6.0
63.3
76.6
32.4
36.3
38.0
Our Methods
18.4
12.2
68.6
80.4
41.2
42.1
43.8
Re-Schedule, sigmoid
18.2
14.0
69.2
81.0
41.5
43.3
44.5
7.3 ABLATION EXPERIMENT
Table Ablation study on tree construction parameters. The default configuration (branching factor k 4, depth d 4) achieves the best performance.
Branch k Depth d
AIME24
AIME25
AMC23
MATH500 Minerva Olympiad
Avg.
4
4
34.2
16.0
71.1
81.8
42.3
44.4
48.3
3
5
33.8
14.8
68.4
79.6
42.3
42.8
46.9
5
3
31.7
14.2
70.4
81.0
41.9
43.0
47.0
We investigate the impact of the reasoning tree's structure by varying the branching factor A: and maximum depth d. The choice of these parameters determines the fidelity of the approximated reasoning tree. While larger values for & and d theoretically provide a more accurate approximation and thus more effective r-score, they also introduce a significant computational overhead. As shown in Table 3, our default configuration of k = 4 and d = 4 yields the best average performance (48.3%). For more detailed analysis, please see Appendices D.3 and D.4
Table Ablation study on the weight function hyperparameters, Wmin and Wmaxx The default setting (0.5, 2.0) performs best.
Wmin
Wmax
AIME24
AIME25
AMC23
MATH500 Minerva Olympiad
Avg.
0.5
2.0
35.2
16.0
72.3
82.2
42.3
44.4
48.5
0.5
1.5
31.4
15.4
72.3
81.8
38.1
42.5
46.9
0.5
3.0
33.5
15.0
69.1
81.8
37.5
41.0
46.3
0.8
2.0
36.6
13.6
71.1
81.6
37.1
43.8
47.3
0.2
2.0
33.5
13.9
71.0
80.0
38.2
41.6
46.4
We analyze the sensitivity of our method to the minimum and maximum Wmax weight hyper- parameters, which control the dynamic range of the curriculum. Results Table 4 show that our default setting of Wiiin = 0.5 and Wimx 2.0 achieves the highest average score (48.5). Decreasing the dynamic range by either reducing Wmax (to 1.5) or increasing Wmin (to 0.8) leads to performance degradation. This indicates that sufficiently large weighting range is erucial for the curriculum to effectively differentiate between easy and hard samples. Conversely, an overly extreme range (c.g., Wiinn 0.2) also degrades performance, possibly because the curriculum excessively under-weights

---

22.67 Published as a conference paper at ICLR 2026
difficult queries. By assigning them minimal weight for a prolonged period, the model is prevented from learning difficult queries. Furthermore, for additional experiments on the design choices for the r-score calculation, please Appendix D.I.
Table 5: Computational cost Performance gain. "Additional Cost" is relative to the total training time.
Tree Size (ka))
33
43
34
4 (Default)
Time Cost (hours)
3.48
6.21
6.70
Additional Cost
+7.45%
+13.30%
+ +14.35%
+48.54% +3.2
Avg Performance Gain
+3.2
+3.0
+4.0
7.4 ANALYSIS EXPERIMENTS
Average performance gain (%)
sigmeid
7.4.1
COMPUTATIONAL COST ANALYSIS
As shown in Figure analyzed the trade-off between the offline tree construction cost and the resulting performance gain.
3.5
T the time cost measured on 8 x H20 GPUs While larger trees (44) incur higher preprocessing costs compared to smaller trees (33), the cost remains manageable relative to the total training time (approx. 46 hours for epochs), and the performance gains are sub- stantial.
2.5
4
y4
4
(+7.45%)
(+13.30%)
(+34.35%
(+48.54%) (+170.84%)
Tree size (Extra cost
Figure 5: Average performance gain versus rea- soning tree size and computational cost.
7.4.2 IMPACT OF ORDERING
Table 6: Comparison between Re-Schedule (Easy-to- and Reverse Schedule (Hard-to-Easy)
Schedule
AIME24
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg
Linear Mapping
Re-Schcdule (Ours)
34.2
15.6
72.4
81.2 81.0
36.4
42.5
47.1
Reverse Schedule
31.9
14.0
67.6
37.8
41.8
45.7
Sigmoid Mapping
Re-Schedule (Ours)
34.2
16.0
71.1
81.8
42.3
44.4
48.3
Reverse Schedule
30.2
15.4
67.1
80.6
34.9
40.2
44.7
To validate the "casy-to-hard" curriculum design, we compared our method against a "Reverse Schedule" where lower r-score (harder) samples are prioritized first. As shown in Table the Re- verse Schedule leads to a significant drop in performance, confirming that starting with structurally simple samples is crucial for effective learning.
CONCLUSIONS
In this work, challenged the reliance path-based metrics for RLVR data scheduling. We introduced the r-score, a novel metric that quantifies learnability based on the structure query's reasoning tree, and proposed Re-Schedule, a curriculum leaming algorithm built upon it. Extensive experiments demonstrated that Re-Schedule consistently outperforms classical RLVR and existing scheduling methods, validating that r-score is more effective proxy for learnability than path-based accuracy. Our findings establish that a structural understanding of the reasoning process provides more powerful and principled foundation for creating efficient training curricula RLVR.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We have manually reevaluated the dataset we created to ensure it is free of any potential for discrim- ination, human rights violations, bias, exploitation, and any other ethical concerns.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our findings, all source code and datasets used in our experiments included in the supplementary material. The provided materials are sufficient to replicate the main results presented in this paper.
REFERENCES
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th anmual international conference machine learning, pp. 41-48, 2009,
Minghan Chen, Guikun Chen, Wenguan Wang, and Yi Yang. Seed-grpe grpo: Semantic entropy enhanced grpo for uncertainty-awan policy optimization. arXiv preprint 1xXi:22505.13366, 2025a.
Xinjie Chen, Minpeng Liao, Guoxin Chen, Chengxi Li, Biao Fu, Kai Fan, and Xinggao Liu. From data-centric to sample-centric: Enhancing Ilm reasoning via progressive optimization. arXiv preprint arXiv:2507.0m 06573, 2025b.
Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang. Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, and Ning Ding. Process reinforcement through implicit rewards. CoRR, abs/2502.014566 2025, doi: 10. 48550/ARXIV22500001556
https://:/ org.10...l0... 48550 arxiv. www.xxvv...02...... 2502. 004.6..0.1466
Yalun Dai, Yangyu Huang, Xin Zhang, Wenshan Wu, Chong Li, Wenhui Lu, Shijie Cao, Li Dong, and Scarlett Li. Data efficacy for language model training. arXiv preprint arXiv:2506.21555. 2025.
Dee,Seek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, and et al Xiao Bi. Deepseek-rl: Incentivizing reasoning capability in Iims via reinforcement learning. CoRR, abs/2501. 2025. doi: 10.48550/ TRXIVV2501.119441
htth:::::: htt.//dc...oo/ wrggr00ggg.. 1...0.8550/2220 arxivv. 2501 12948
Jiaxuan Gao, Shusheng Xu, Wenjie Ye, Weilin Liu, Chuyi He, Wei Fu, Zhiyu Mei, Guangju Wang, and Yi Wu. On designing effective RL reward at training time for LLM reasoning. CoRR, doi: 10.4850/ARXI.22100.555
abs/2410.15115, 2024.
URL https:: orq/10.
18550/arXiva 25115
Yanı Hao, Li Dong, Xun Wu, Shaohan Huang, Zewen Chi, and Furu Wei. On-policy with optimal reward baseline. arXiv preprint 1rXivv2500523588 2025a.
Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. Rethinking entropy interventions in rlyr: An entropy change perspective. arXiv preprint A.1.:225101115555 2025b.
Zhezheng Hao, Hong Wang, Jian Luo, Jianqing Zhang, Yuyan Zhou, Qiang Lin, Can Wang. Hande Dong, and Jiawei Chen. Recreate: Reasoning and creating domain agents driven by experience. arXiv preprint arXiv:2601.11100, 2026.
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Pro- ceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL 2024, Bangkok, Thailand, August 11-16, 2024. pp. 3828 3850. Asso- ciation for Computational Linguistics, 2024, doi: acl 221 211
URL
https: :ttps://doii http://tf..ooo/.//// /tt.g//00. 0. 1Tw.1..1.1111111 16653/v1/2024

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Joaquin Vanschoren and Sii-Kit Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks
I, NeurlPS Datasets and Benchmarks 2021, December 2021, virtual, 2021. URL
www.eeeddiinn...euissssse...e.ee wttpssscpppapp/2/.//////// roundo.
0u55:/2833333cc000777/2222220213 -beiaaltbbsttatttcc Sobnstract-ound...te html
Jian Hu. REINFORCE+:: A simple and efficient approach for aligning large language models.
CoRR, abs/2501 032622, 2025. doi: 10.48550/ARXIV.5000 033662. ht.htt:p:: /doi. /rg/
10. 7..48550/axxiv 2501 2501.03262
Jingcheng Hu, Yinmin Zhang. Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung- Yeung Shum. Open-reasoner-zzeo: An open source approach to scaling up reinforcement learning on the base model, 2025. URL hoRl:httss://axxv.ccoo
http:///aa////0..3 24290
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.0799 079744 2024.
Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron C. Courville, and Nicolas Le Roux. Vineppo: Unlocking RL potential for LLM reasoning through refined credit assignment. CoRR, abs/2410.01679, 2024. doi: 10. (0.48550/ARXIV.2410) htt.://............5//////////// www.xxiv.2111.....
01679. URL httpss://dol /ttp:////ooooo//1.//// 48550/
Kimi, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang. Hao Zhang, Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haozhen Yu, Hongcheng Gao, Huabin Zheng, Huan Yuan, Jia Chen, Jianhang Guo, Jianlin Su, Jianzhou Wang, Jie Zhao, Jin Zhang. Jingyuan Liu, Junjie Yan, Junyan Wu, Lidong Shi, Ling Ye, Longhui Yu, Mengnan Dong, Neo Zhang, Ningchen Ma. Qiwei Pan, Qucheng Gong, Shaowei Liu, Shengling Ma, Shupeng Wei, Sihan Cao, Siying Huang, Tao Jiang, Weihao Gao, Weimin Xiong, Weiran He, Weixiao Huang, Wenhao Wu, Wenyang He, Xianghui Wei, Xianqing Jia, Xingzhe Wu, Xinran Xu, Xinxing Zu, Xinyu Zhou, Xuchai Pan, Y. Charles, Yang Li. Yangyang Hu, Yangyang Liu, Yanru Chen, Yejie Wang, Yibo Liu, Yidao Qin, Yifeng Liu, Ying Yang. Yiping Bao, Yulun Du Yuxin Wu, Yuzhi Wang, Zaida Zhou, Zhaoji Wang. Zhaowei Li, Zhen Zhu, Zheng Zhang, Zhexu Wang, Zhilin Yang. Zhiqi Huang, Zihao Huang, Ziyao Xu, and Zonghan Yang. Kimi k1.5: Scaling reinforcement learning with llms. CoRR, abs/2501.12599, 2025. doi: 10. 48550/ARXIV.2501 12599. URL htt.hhttss//// /ooi worg/1....0...oooooooooo 48550885500 http//awww...............
Weixian Lei, Jiacong Wang, Haochen Wang, Xiangtai Li, Jun Hao Liew, Jiashi Feng, and Zilong Huang. The scalability of simplicity: Empirical analysis of vision-language learning with single transformer. arXiv preprint arxiv:2504. 10462, 2025.
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with lan- guage models. In Sanmi Koyejo. S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural In- formation Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 De- cember 9, 2022, 2022. htt: http::://pppprrsseee nips cc httpcc///pppeeeeeee http://aarfffeessseeepppp wwt.:ffaapeeeeee/////// hbat://aaberffcee2211ddddo0ooo.3 -bt.:/aaseeeeeel---aaataaaaaaeae Ceneeettooooeeeeeeeee htmml
Jia Li. Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9, 2024. http: Publications numina dataset.
bicmr bbi.mr.kuu.c.. edu. ww/.cn///ooo cno-diigbnn/
Xuefeng Li, Haoyang Zou, and Pengfei Liu. LIMR: less is more for RL scaling. CoRR, abs/2502. 1 1886, 2025. doi: 10448550/ARXIV.55022.1 1886. URL htttps:///aa http:///ooo..i//.../ 4 48550/ 2502. 11886
12

---

Published as a conference paper at ICLR 2026
Weiting Liu, Han Wu, Yufei Kuang, Xiongwei Han, Tao Zhong, Jianfeng Feng, and Wenlian Lu. Automated optimization modeling via a localizable error-driven perspective. arXiv preprint 12XYv:2602.11164, 2026.
Zichen Liu, Changyu Chen, Wenjun Li. Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl rl-zero-like training: A critical perspective. CoRR, abs/2503.207883, 2025. doi: 10.48550ARXXV..555.22200 207888.. 20783
Jun-Yu Yu Ma, Hong Wang, Hao-Xiang Xu, Zhen-Hua Ling, and Jia-Chen Gu. Perturbation-restrained sequential model editing. In The Thirteenth International Conference Learning Representa- tions.
Xingyu Ren, Pengwei Liu, Pengkai Wang, Guanyu Chen, Qinxin Wu, and Dong Ni. Uncertainty- informed meta pscudo labeling for surrogate modeling with limited labeled data. In The Thirty- ninth Annual Conference on Neural Information Processing Systems.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, 0a//707.006477 2017. URL http::/arxiv. org.abs/
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li,
Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300 2024. doi: 10.48550/ARXXV.2440.0000 https://doi wtw./1..../0..00001. /ttp/aaiiivv.. 2402.24400.0 03300
URL
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference Computer Systems, pp. 1279-1297, 2025.
Mingyang Song, Mao Zheng, Zheng Li, Wenjie Yang. Xuan Luo, Yue Pan, and Feng Zhang. Fastcurl: Curriculum reinforcement learning with progressive context extension for efficient train- ing ri-kike reasoning models. CORR, abs/2503 17287, 2025. doi:
URL https:://oo...... http://ti..oogo//// 40w.0004450000100 2503.
Haochen Wang. Xiangtai Li, Zilong Huang, Anran Wang, Jiacong Wang, Tao Zhang, Jiani Zheng, Sule Bai, Zijian Kang, Jiashi Feng, et al. Traceable evidence enhanced visual grounded reasoning: Evaluation and methodology. arXiv preprint arXiv:2507.07999, 2025a.
Haochen Wang, Yuhao Wang, Tao Zhang, Yikang Zhou, Yanwei Li, Jiacong Wang, Jiani Zheng. Ye Tian, Jiahao Meng. Zilong Huang, et al. Grasp any region: Towards precise, contextual pixel understanding for multimodal 1lms. arXiv preprint wxXi:2510.188777 2025b.
Jiacong Wang, Bohong Wu, Haiyong Jiang, Xun Zhou, Xin Xiao, Haoyuan Guo, and Jun Xiao. World to code: Multi-modal data generation via self-instructed compositional captioning and filtering, arXiv preprint 2il::22492222244 2024.
Jiacong Wang. Zijian Kang, Haochen Wang, Haiyong Jiang, Jiawen Li, Bohong Wu, Ya Wang. Jiao Ran, Xiao Liang, Chao Feng, et al. Vgr: Visual grounded reasoning. arXiv preprint arXiv:2506.1199) 2025c.
Jiakang Wang, Runze Liu, Fuzheng Zhang, Xiu Li, and Guorui Zhou. Stabilizing knowledge, pro- moting reasoning: Dual-token constraints for rlvr. arXiv preprint arXiv:2507. 15778, 2025d.
Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for Ilm reasoning. arXiv preprint wxXi:2506.019333 2025e.
Chenxing Wei, Mingwen Ou, Ying He, Yao Shu, and Fei Yu. PAFT: Prompt-agnostic fine- tuning. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro- cessing, pp. 699--117 Suzhou, China, November 2025a. Association for Computational Lin- guistics. ISBN 779-88899176-322-4 a/aclanthol ogy,
doi: 10. 10.18653/v (t2.8655/111222.mmmmllm..
URL https:
emn ppm wa..mann.31.
13

---

Published as a conference paper at ICLR 2026
Chenxing Wei, Yao Shu, Ying Tiffany He, and Fei Yu. Flexora: Flexible low-rank adaptation for large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Moham- mad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Com- putational Linguisties \Volume I: Long Pupers), pp. 14643-14682 Vienna, Austria, July 2025b. Association for Computational Linguistics. ISBN 979-8-89176--51--,, doi: 10. 10.18653/v1/2025.
acl-long.710 713. URL htttt:t//accaaateeeee http:/cccantthoooogoooo wtg.
long. 713/
Chenxing Wei, Jiarui Yu, Ying Tiffany He, Hande Dong, Yao Shu, and Fei Yu. Redit: Reward dithering for improved L.LM policy optimization. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025c. httphttsss//pperevvviteee http:/eiwwwiewwfoorrreeeeee formm?
id-pG11 PG1G6633MMa
Yangbo Wei, Li Huang, Qi Feng, Zhanfei Chen, Jinlong Yan, Ting-Jung Lin, Zhen Huang, Kun Ren, Wei Xing, and Lei He. Modelgen: Automating semiconductor parameter extraction with large language model agents. ACM Transactions on Design Automation of Electronic Systems, 30(6): 1-26, 2025d.
Yangbo Wei, Zhen Huang, Huang Li, Wei Xing, Ting-Jung Lin, and Lei He. Vflow: Discovering optimal agentic workflows for verilog generation. arXiv preprint arXhv:2504.03723, 2025e.
Liang Wen, Yunke Cai, Fenrui Xiao, Xin He. Qi An. Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang. Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. Light-rl: Curriculum sft, DPO and RL for long COT from scratch and beyond. CoRR, abs/2503. 10460,, 2025. doi: (0448550/RXXI.2550..0ee 10460. URL htw.t:ttp////c... 48550 wwt.///axvvv... 2503. 10000..10466 10460
org/10.
Zhiheng Xi, Wenxiang Chen, Boyang Hong, Senjie Jin, Rui Zheng, Wei He, Yiwen Ding, Shichun Liu, Xin Guo, Junzhe Wang, Honglin Guo, Wei Shen, Xiaoran Fan, Yuhao Zhou, Shihan Dou, Xiao Wang, Xinbo Zhang, Peng Sun, Tao Gui, Qi Zhang, and Xuanjing Huang. Training large language models for reasoning through reverse curriculum reinforcement learning. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https: hettps://opezzeeee wwtp//wenneeeeeeeepeeeee...eeeee wwt. htt.:/ooommmddddeeee
Qingyue Yang. Jie Wang, Xing Li, Zhihai Wang, Chen Chen, Lei Chen, Xianzhi Yu, Wulong Liu, Jianye Hao, Mingxuan Yuan, et al. Attentionpredictor: Temporal pattern matters for efficient 11m inference. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a.
Qingyue Yang, Jie Wang, Xing Li, Yinqi Bai, Xialiang Tong, Huiling Zhen, Jianye Hao, Mingxuan Yuan, and Bin Li. Why attention patterns exist: A unifying temporal perspective analysis, 2026.
Zhicheng Yang, Zhijiang Guo, Yinya Huang, Xiaodan Liang, Yiwei Wang, and Jing Tang. Treerpo: Tree relative policy optimization. arXiv preprint arXiv:2506.05 05.0000555/... 183, 2025b.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. DAPO: open- source LLM reinforcement learning system at scale. CoRR, abs/2503. 14476. 2025. doi: 10.
48550/ARXIV.25003 14476. URL /oo.. wrgg10...0.... 48550/ arwiv.iiiiv...... 200....4500344 14476
Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. What's behind ppo's collapse in long-cot? value optimization holds the secret. CoRR, abs/2503.01491, 2025. doi: 10.48550/
ARXIV.2503.0148 URL (RRL:hhtss:///// /ttp://////o....../.oo.oo/ 111501.4455//aa.
01491
Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, Tian Fan, Zhengyin Du, Xiangpeng Wei, Xiangyu Yu, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Ru Zhang, Xin Liu, Mingxuan Wang, Yonghui Wu, and Lin Yan. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks, 2025. URL httph::/ppa..tt... org//bb/ 2504. 05118
14

---

Published as a conference paper at ICLR 2026
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- 20o: Investigating and taming zero reinforcement learning for open base models in the wild. CoRR, abs/2503 18892, 2025. doi: 10.48550ARXXIV.353..1888
[tthts::://aaa org/
10. 10.48550/arXiv 25033 18892
Xiaojiang Zhang, Jinghui Wang, Zifei Cheng, Wenhao Zhuang, Zheng Lin, Minglei Zhang, Shao- jie Wang, Yinghan Cui. Chao Wang, Junyi Peng. al. Srpo: cross-domain implementation of large-scale reinforcement learning on 1lm. arXiv preprint arXivv2504.14286, 2025a. URL
httpss://axiiv. org/ abs 2504 14286
Xinnan Zhang, Chenliang Li, Siliang Zeng, Jiaxiang Li, Zhongruo Wang, Kaixiang Lin, Songtao Lu, Alfredo Garcia, and Mingyi Hong. Aligning frozen Ilms by reinforcement learning: An iterative swweightthennootimizz approach. arXiv preprint arXiv:2506. 17828, 2025b.
Fei Zhao, Chonggang Lu, Zheyong Xie, Ziyan Liu, Haofu Qian, Jianzhao Huang, Fangcheng Shi, Zijie Meng, Hongcheng Guo, Mingqian He, et al. Redone: Revealing domain-specific Ilm post- training in social networking services. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 2648 2674, 2025.
15

---

Published as a conference paper at ICLR 2026
A USAGE OF LLMS
Throughout the preparation of this manuscript, Large Language Models (LLMs) were utilized writing and editing tool. Specifically, we employed LLMs to improve the clarity and readability of the text, refine sentence structures, and correct grammatical errors. All final content, including the core scientific claims, experimental design, and conclusions, was conceived and written by us, and we take full responsibility for the final version of this paper.
B ACKNOWLEDGEMENTS
The authors would like to thank all the anonymous reviewers for their insightful comments and valuable suggestions. This research is supported by Anhui Provincial Natural Science Foundation (Grant No. 2002248888QF2144 the Fundamental Research Funds for the Central Universities (Grant No. WK208000000) and the Opening Project of the State Key Laboratory of General Artificial Intelligence (Project No.SKLAAG22250P06) The work is also sponsored by the CodeBuddy (https://ww..ood)) http://www....ddd..i..o.o.o Special thanks go to Yi Liu and the team at Tencent for their gen- erous support and assistance.
C DETAILS OF EXPERIMENTAL SETUP
All algorithms are implemented based on the official GRPO codebase within the VeRL framework. We use learning rate of le-6 without warm-up across all experiments. At each rollout step, we generate 8 answers for each of 512 sampled questions, then split the data into 16 mini-batcher and train the policy network for 16 gradient steps. Models are trained for at most 150 rollout steps. Unless otherwise specified, we follow GRPO's default design choices with token-level loss normal- ization without dynamic sampling and KL regularization. For all models, the maximum input length is 1024 and the minimum input length is 3072. All the experiments were conducted H20 GPUs.
Note: The authors of Eurus-PRIME only published results from training on Qwen2.5-Math, Therefore, do not include results for the Qwen2.5-7B model in comparison.
D SUPPLEMENTARY EXPERIMENT
D.1 EFFECT OF METRIC SELECTION
Table 7: Ablation study comparing our proposed node-level modification metric ("Fix') with branch-level 'Pruning' metric. "Fix" consistently outperforms 'Pruning', validating our fine-grained approach.
Model
AIME24 AIME25 AMC23 MATH500 Minerva Olympiad Avg.
Fix
Re-Scheduleiinearr
34.2
15.6
72.4
81.2
36.4
42.5
47.1
Re-Schedulc, S--Schduulesgmoo.. Eeigmoöd
34.2
16.0
71.1
81.8
42.3
44.4
48.3
Pruning
Re-Schedullcynaar
35.7
14.6
73.7
81.0
34.9
41.6
46.9
Re-Schedule Se-Schhedulesggmodd sigmoid
33.1
16.7
71.1
82.0
39.0
42.4
47.4
We validate our core design choice for the r-score calculation. Our proposed method ('Fix") defines an 'edit' as a single node modification. We compare this against alternative ("Pruning'), where an 'edit' is defined as pruning an entire sub-branch from a decision point. Table 7 shows that the Fix' method consistently outperforms 'Pruming' for both linear (47.15 1% 46.9% and sigmoid (48.3% vs. 47.4% schedules. This result shows that compared with the branch 'Pruning', the node 'Fix is more consistent with the training process of reinforcement learning.
16

---

Published as a conference paper at ICLR 2026
Table 8: Variance of Re-Schedule over multiple runs.
AIMEE2
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg. + Var.
35.2 + 0.1
16.0 + 0.0
72.3 = 0.4
82.2 + 0.0
42.3 0.6
44.4 + 0.6
D.2 VARIANCE ANALYSIS
To assess the stability of proposed Re-Schedule method, conducted repeated experiments using different random seeds. rrepots the variance. The results demonstrate that our method exhibits low variance. This confirms that the reported improvements are statistically stable and not due to random variation. Compared to performance improvements, the impact of variance is minimal.
D.3 ABLATION ON TOKEN INTERVAL !
To investigate the impact of I on performance and relationship with model capabilities, we con- ducted ablation studies on two base models: Qwen2.5-Mathe and Qwen2.5-7B. We utilized the Sigmoid weighting mapping for these experiments.
Table summarizes the average accuracy across six benchmarks (AIME24, AIME25, AMC23, MATH500, Minerva, Olympiad) for varying token intervals {200, 400, 600, 12003.
Table 9: Impact of Token Interval on Average Accuracy (Sigmoid Mapping).
Interval (l)
Qwen2.5-Math-78 (Avg)
Qwen2.5-7B (Avg)
1 200
48.3
44.5
= 400
48.6
43.2
7 600
48.9
43.1
= 1200
46.0
41.1
The results indicate that Re-Schedule is generally robust to the choice of within the range of [200, 600]. For the specialized math model (Qwen2.5-Math-7B) performance remains high and stable as increases to 600. For the general model (Qwen2.5-7B), while = 200 yields the best results, the performance drop at I 600 is relatively contained. A significant performance drop observed for both models when = 1200. This suggests that when the interval is too large, the approximated reasoning tree becomes coarse to capture the critical branching points necessary for effective r-score estimation.
D.4 SENSITIVITY BRANCHING FACTOR k DEPTH d AND MODIFICATION BUDGET M
We investigated the impact of the reasoning tree size on performance by varying the branching factor l and depth d the Qwen2.5--ath- model.
Table 10: Ablation study on branching factor (with fixed d = 4).
Setting
AIME24
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg
Linear Mapping
k 3
32.5
15.2
74.0
81.8
36.3
41.0
46.8
(Default)
34.2
15.6
72.4
81.2
36.4
42.5
47.1
k 5
35.1
18.1
77.4
81.8
37.8
44.6
49.1
Sigmoid Mapping
k 3
32.0
15.2
74.2
81.6
38.6
43.1
47.5
k 4 (Default)
34,2
16.0
71.1
81.8
42.3
44.4
48.3
k 5
36.4
17.0
75.5
81.2
40.6
43.4
49.0
17

---

Published as a conference paper at ICLR 2026
Table 11: Ablation study on tree depth d (with fixed 4).
Setting
AIME24
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg
Linear Mapping
= 2
31.1
15.3
74.2
81.8
38.2
42.2
47.1
d 3
31.4
14.6
72.7
82.0
39.7
43.3
47.3
(Default)
34.2
15.6
72.4
81.2
36.4
42.5
47.1
Sigmoid Mapping
d = 2 d 3 d = 4 (Default)
31.9
14.8
74.5
81.6
37.2
42.4
47.1
33.2
16.4
73.0
80.0
41.9
41.6
47.7
34.2
16.0
71.1
81.8
42.3
44.4
48.3
Table 12: Ablation study on node modification budget M (with fixed k = 4, 4,d= = 4,1 200).
Setting
AIME24
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg
M
33.6
15.4
72.2
79.0
40.4
43.1
47.3
M 10 (Default)
34.2
16.0
71.1
81.8
42.3
44.4
48.3
M 15
34.7
16.0
71.8
82.0
41.6
42.4
48.1
Varying branching factor k: Fixing = and = 200, we tested l E (3,4,5). As shown in Table I0 increasing & generally improves performance, suggesting that denser tree captures the structural difficulty more accurately.
Varying tree depth d: Fixing k 4 and I 200, tested d (2,3,4) 4). Table 11 shows that deeper trees provide a better estimation of the reasoning structure, leading to improved downstream performance.
Varying node modification budget M: Finally, we assess the stability of our method with respect the node modification budget M. Fixing 4, d and 200, evaluated performance across M {5, 10, 15}. As presented in Table 12 the results are relatively robust to changes in this parameter. While the default setting of M 10 yields the optimal average accuracy, varying the budget between and 15 results in no significant performance degradation, indicating that the r-score remains reliable metric across different budget constraints.
D.5 GENERALIZATION TO DIFFERENT MODEL ARCHITECTURES
To demonstrate the broad applicability of method beyond the Qwen2.5 family, conducted additional experiments on the Qwen3-4B-BB model.
Table 13: Performance comparison on Qwen3-4B-Ban
Model
AIME24
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg
GRPO
24.2
21.8
52.4
86.0
39.4
43.4
44.5
ACC
24.8
23.3
59.3
88.8
41.6
42.0
46.6
Re-Schedule (Ours)
27.6
26.9
57.6
89.8
43.5
47.4
48.8
Re-Schedule consistently outperforms both the standard GRPO baseline and the accuracy-based curriculum (ACC) across all benchmarks. This confirms that the effectiveness of the r-score limited specific model architectures sizes.
As shown in Table 13
18

---

Published as a conference paper at ICLR 2026
D.6 DYNAMIC R-SCORE CALCULATION
To determine if the r-score should be updated as the model evolves, we compared our standard static approach (computed once before training) with dynamic approach where the r-score is computed and weights are updated three times during the training process.
Table 14: Comparison between Static and Dynamic R-Score updates on Qwen2.5-Math-7.
Method
AIME24
AIME25
AMC23
MATH500
Minerva
Olympiad
Avg
Linear Mapping
Static (Default)
34.2
15.6
72.4
81.2 80.0
36.4
42.5
47.1
Dynamic (3 updates)
34.6
14.9
75.3
39.7
41.6
47.8
Sigmoid Mapping
Static (Default)
34,2
16.0
71.1
81.8
42.3
44.4
48.3
Dynamic (3 updates)
35.3
15.2
74.2
82.4
42.7
43.6
48.9
As shown in Table 14| the dynamic approach yields performance comparable to the static baseline (e.g., 48.9% vs. 48.3% for Sigmoid mapping). Given the substantial computational cost of re- generating reasoning trees during training, we conclude that the static r-score serves as a sufficient and efficient prior for guiding the curriculum.
19

---

Published as a conference paper at ICLR 2026
D.7 GENERALIZATION TO CODE GENERATION
Table 15: Performance comparison on Code Generation (LiveCodeBench v5).
Method
pass
pass @ 4
GRPO
25.4
35.4
ACCsigooid Re-Schedule sigmoid
25.8
36.0
26.3
37.8
To validate the generalization capability of Re-Schedule beyond mathematical reasoning, ex- tended our evaluation to the domain Code Generation. We utilized DeepSeek-Ri. RI---itiill----- Qwen- 1.5B as the base model. The model was trained
the ArcherCodeR dataset Wang et al. (2025d),
which contains 6,753 code generation tasks. For evaluation, we used the LiveCodeBench v5 bench- mark Jain et al.
2024 We report pass @1 1 and pass @ 4 metrics (averaged over 8 samples).
As shown in Table 15 Re-Schedule consistently outperforms both the standard GRPO baseline and the accuracy- beauea curriculum (ACC). Specifically, method achieves +0.9% improvement in pass @ and significant +2.4% improvement in pass@4 compared to GRPO. These results confirm that the structural insights captured by the r-score effective in the coding domain, where the "reasoning tree" corresponds to the decision space of code logic and syntax. Looking forward. the tree-based structural metries could also be adapted improve multi-modal data generation pipelines
and self- self-instructed compositional code captioning (Wang et al. 2024 2025b
Hao et al. 2026
Wang et al. 2025a
Lei al. 2025).
20


Reviews:
[{"review_id": "LZG3sDNaGc", "reviewer": "Reviewer_3KpD", "summary": "This work proposes Reasoning Tree Schedule (Re-Schedule), a scheduling algorithm for reasoning tasks that constructs a curriculum from structurally simple to complex problems. The complexity is defined as r-score, a metric that measures the difficulty for learning based on the structure of the reasoning tree. Experiments on six math benchmarks show that Re-Schedule achieves better performance than existing methods, with an average improvement of up to 3.2%.", "questions": "* Are results obtained with averaging over multiple runs? If not, the authors are encouraged to provide additional experiments with averaging to make the results more stable. This affects the validity of the method and is an important factor of the score and recommendation.\n* Does the optimal token interval depend on the model type and the difficulty of the question set (e.g., Qwen2.5-7B-Math vs Qwen2.5-7B, GSM8k vs DAPO training set)?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* This work introduces a novel metric, r-score, to measure the difficulty of reasoning tasks in terms of the structure of the reasoning tree. The proposed metric shifts the focus from only accuracy to structure-based difficulty assessment.\n* With the proposed metric, the authors construct a curriculum from structurally simple to complex problems. This method is potentially applicable to other tasks and domains.\n* The curriculum is used to train a reasoning model, which shows improved performance on six math benchmarks.\n* The ablation studies show the importance of the proposed hyperparameters.", "weaknesses": "* Missing detailed analysis of the variation of the results. For example, since AIME 24/25 only has 30 problems each, the results may be not stable. The variation can be more than the 1.7% improvement on Qwen2.5-7B in Table 2.\n* Adding to the point above, while prior works such as SimpleRL-Zoo (which was cited in Table 1 and 2) uses metrics such as Avg@32 to make the result more stable, the authors did not mention whether averaging is performs for Re-Schedule. If not performed, the results may be not stable.\n* This work performs experiments with a fixed token interval (l=200). However, no ablations are performed to investigate the impact of the token interval, which determines how coarse the reasoning tree is.\n* Comparisons with prior works are not performed in a fair way. For example, SimpleRL-Zoo makes use of GSM8k and MATH datasets only, while this work makes use of DAPO training set. Since the proposed method is a training schedule algorithm, the comparison should be performed in the same training set with the baseline (or with a reproduced result using the DAPO training set).\n* Authors did not provide theoretical analysis of the proposed method. The proposed node-editing motivation is intuitive, but not formally justified or validated beyond empirical observation in Figure 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T4vH2rTzmz", "reviewer": "Reviewer_qoyQ", "summary": "This paper focuses on designing a more effective data scheduling strategy that leverages the structural relationships among data samples through reasoning trees. The proposed R-score captures these internal structural dependencies, while traditional path-based metrics such as accuracy fail to do so. Building on this metric, the Re-Schedule algorithm prioritizes high R-score samples early in training, using curriculum learning principles to balance exploration and exploitation and enhance generalization.", "questions": "1. In Figure 3(a), do the illustrated parameters correspond to k,d=3? It would be helpful to align the figure with the numerical example in Lines 256–257 for consistency.\n2. What would happen if training started with lower R-score (i.e., more difficult) samples? Would the model still converge, and how would this affect generalization?\n3. Since the evaluation temperature is set to 1.0, which introduces more randomness, how many evaluation runs were averaged? Are the reported improvements statistically significant?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The reasoning tree formulation is intuitive and well motivated. The authors clearly articulate the limitations of path-based metrics and provide convincing evidence supporting the structural view of reasoning. Empirically, the results demonstrate consistent improvements: both Qwen2.5-Math-7B and Qwen2.5-7B models outperform all baselines across multiple mathematical reasoning datasets. The ablation studies and supporting analyses further validate the soundness of the approach.", "weaknesses": "1. Since reasoning trees can grow exponentially with depth, the authors approximate them using a fixed structure defined by (k,d,l). While this keeps the computation tractable, it may oversimplify the actual reasoning process. Although ablation studies explore nearby settings, all tested configurations remain within the same order of magnitude. How sensitive is performance to these parameters? Would larger or deeper trees provide further gains? Or would they introduce diminishing returns? Some intuition for choosing these parameters in advance would be valuable.\n2. The paper does not fully discuss the computational cost of sampling k^d branches, even with KV-cache reuse. A quantitative analysis of runtime or memory cost would help assess scalability to larger models or non-mathematical reasoning tasks.\n3. Writing and clarity: In Section 4, the meaning of the blue line in Figure 2 is unclear. Does it correspond to queries with more complex reasoning structures? Likewise, the terms potential samples and stagnant samples are not well explained in the caption. Clarifying these details would improve readability.\n4. The R-score metric is intuitive but relies on several hyperparameters \\eta,w_{\\min},w_{\\max},k,d,l, which may need careful tuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6EsND1VWhM", "reviewer": "Reviewer_Hh8P", "summary": "This paper challenges the use of path-based accuracy for RLVR data scheduling, arguing it is a poor proxy for true learning difficulty. The authors reframe the problem around the \"Reasoning Tree\" structure, proposing the \"r-score\": a novel metric that quantifies a query's \"learning potential\" based on the maximum accuracy gain achievable under a limited \"node editing budget\". They introduce \"Re-Schedule\", a curriculum learning algorithm that uses the r-score to prioritize structurally simple queries first. Experiments on six math benchmarks show that Re-Schedule significantly outperforms strong baselines, achieving state-of-the-art results.", "questions": "3.  High Computational Overhead: The method relies on a substantial offline pre-computation stage to build approximate trees and calculate r-scores for all queries, which may be computationally infeasible for very large-scale datasets.\n\n4.  Hyperparameter Sensitivity: The r-score's quality depends on tree approximation hyperparameters ($k$, $d$, $w$). As Table 3 and Table 4 suggests, performance is sensitive to these choices, but the paper lacks a principled guide for setting them.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.  Insightful Problem Definition: The paper provides a valuable reframing of the scheduling problem, moving from simple accuracy to the more nuanced \"structural complexity\" of a query's reasoning tree. The \"node-editing\" analogy is highly intuitive.\n\n2.  Principled Metric (r-score): The r-score is a well-defined metric that attempts to directly quantify \"learning potential\" rather than just static difficulty. This is a more principled approach to capturing the learning dynamics of a query.\n\n3.  Strong Empirical Results: The Re-Schedule method achieves state-of-the-art results across six math benchmarks. The significant accuracy gains (up to 3.8%) over strong RLVR and scheduling baselines compellingly demonstrate the method's effectiveness.", "weaknesses": "1.  Limited Experimental Domain: The method's effectiveness is only validated on math-reasoning tasks. It is unclear if the r-score framework will generalize to other domains. Code generation would be a good benchmark with verifiable rewards.\n\n2.  Insufficient Discussion of Related Work: The paper needs to better position its contributions relative to the broader curriculum learning (CL) field. A deeper comparison with other methods that also estimate 'learning potential' (e.g., via gradients or uncertainty) is warranted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wnMyYexg5f", "reviewer": "Reviewer_eZz4", "summary": "This paper introduces R-Score, a novel metric to quantify the learnability of queries in RL. R-Score measures how many node modifications are required in a reasoning tree to achieve a correct answer, reflecting how suitable a sample is for training.\nIn experiments, curriculum learning based on R-Score achieves better performance than accuracy-based and random scheduling methods, demonstrating that R-Score can effectively guide sample selection during RL training.", "questions": "Is the R-Score static during training, or do you recompute it as the model improves?\nSince the model’s competence evolves, a query that was previously hard may later become easy. Would it be beneficial or necessary to periodically update the R-Score to reflect this changing learnability?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a new metric, R-Score, to evaluate whether a question is appropriate for RL training. The design aligns well with the intuition that queries that require fewer reasoning corrections are easier to learn, and the empirical results consistently support this idea.\n\n2. The paper is clearly written, with detailed explanations of the R-Score computation, experimental setup, and ablation studies. Figures and tables effectively illustrate the trends and improvements.", "weaknesses": "1. The method is only validated on the Qwen2.5 and math task, which is a special combination that makes the results less general [1]. To establish broader applicability, more models and diverse reasoning tasks should be included.\n\n2. The computation of R-Score depends heavily on several hyperparameters such as the branching factor, depth, and node modification budget. These choices can dramatically increase computational requirements, making large-scale use impractical without further optimization or approximation.\n\n[1] Wu, Mingqi, et al. \"Reasoning or memorization? unreliable results of reinforcement learning due to data contamination.\" arXiv preprint arXiv:2507.10532 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)