MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model
Authors: Prasanna Mayilvahanan
Ricardo Olmedo
Thaddäus Wiedemer
Wieland Brendel
Venue: ICLR 2026 Poster (2026)
Keywords: RLVR
RL
Reasoning
Math
LLM Evaluation
Primary area: datasets and benchmarks
TL;DR: MATH-B is a zero-baseline benchmark—base models achieve $\texttt{pass@1024}\approx 0$—designed to drive exploratory RL methods that genuinely expand reasoning beyond base capabilities.

Abstract: With the advent of DeepSeek-R1, a new wave of reinforcement learning (RL) methods has emerged that seem to unlock stronger mathematical reasoning. However, a closer look at the open-source ecosystem reveals a critical limitation: with sufficiently many draws (e.g., $\texttt{pass@1024}$), existing base models already solve nearly all questions on widely used math benchmarks such as MATH-500 and AIME 2024. This suggests that the RL fine-tuning methods prevalent in the LLM reasoning literature largely sharpen existing solution modes rather than discovering entirely new ones. Such sharpening stands in contrast to the broader promise of RL: to foster exploration and to acquire new skills. To move beyond this plateau, we introduce MATH-Beyond (MATH-B), a benchmark deliberately constructed to defeat common open-source models of up to 8B parameters even under large sampling budgets. Improving performance on our benchmark via RL requires methods that learn to reason in ways that go beyond base model capabilities in repeated sampling. Since the problems are drawn from subsets of DAPO-Math-17K and DeepScaleR datasets, they remain topically equivalent to standard high-school math. Validating our premise, RL fine-tuned models such as Nemotron-Research-Reasoning-Qwen-1.5B and DeepScaleR-1.5B-Preview perform poorly on MATH-B at $\texttt{pass@1024}$, showing how existing approaches fall short on tackling harder instances. We hope MATH-B will catalyze exploration-driven RL approaches that elicit deeper reasoning capabilities.

Published as a conference paper at ICLR 2026
MATH-BEYOND: A BENCHMARK FOR RL TO EXPAND BEYOND THE BASE MODEL
Prasanna Mayilvahanan' 122,3
Ricardo Dominguez-Olmetto
Thaddäus Wiedemer1
Wieland Brendel
ABSTRACT
With the advent of DeepSeek-RI, a new wave of reinforcement learning (RL) methods has emerged that seem to unlock stronger mathematical reasoning. How- ever, closer look at the open-source ecosystem reveals critical limitation: with sufficiently many draws (e.g.. pass 02 4), many existing base models already solve nearly all questions on widely used math benchmarks such as MATH-500 and AIME 2024. This suggests that the RL fine-tuning methods prevalent in the LLM reasoning literature largely sharpen existing solution modes rather than dis- covering entirely
ones. Such sharpening stands in contrast the broader promise of RL: to foster exploration and to acquire new skills. To move beyond this plateau, introduce MATH-Beyond (MATH-B), benchmark deliberately constructed to defeat common open- source models of up to 8B parameters even under large sampling budgets. Improving performance on our benchmark via RL re- quires methods that learn to reason in ways that go beyond base model capabilities repeated sampling. Since the problems are drawn from subsets of DAPO- Math-17K and DeepScaleR datasets, they remain topically equivalent to standard high-school math. Validating our premise, RL fine-tuned models such as Nemotron- Research-Reasoninn Baaasoninn--wen--.... Q.en-1.. and DeepScaleR- 1.5.S Preview perform poorly on MATH-B at pass@10 1024, showing how existing approaches fall short on tack- ling harder instances. We hope MATH-B will catalyze exploration-drive RL ap- proaches that elicit deeper reasoning capabilities. We release MATH-B at https: / //huggi ng face. co hts//attassettee htweee//bbbeeeeee www.dde/eaaoooooorMrrHaHa Reyond.
Datasets
Quality
Pass@1 1024 Filtering
MATH-Beyond
Dwan2.5
77%
81%
Owend.d Math
Owen2.5-7
QBA Type Filtaring
181
Owerö
Passig 1024
BArr--Mrth11 1/X
Phe screrrngg
0L3661/2
DospSouldR
GT Visificttion
41
Usma:3311
Dedupiocation
R1 Disii Owen
-0% Passeg 1004
AIME24
MATH-BByyon
Figure MATH-Beyond: Benchmark Construction and Difficulty. Left: Schematic of the MATH-B creation process. A large set of problems from DAPO-Math-17K and DeepScaleR is first refined through quality filters ensure answer correctness and verifiability. This followed by evaluation against a gauntlet of open-source base models (< 8B, e.g., Qwen3, Qwen2.5 (-Math), DeepSeek-RR ---isssil a passt 024 budget to isolate problems that lie beyond their limits. The filtering yields the MA suite of benchmarks: 41 -problle intersection set (unsolved by all base models) for evaluating universal difficulty, and a larger 181 -probbee union set (unsolved by least one model) with model-specific splits for targeted analysis. This suite provides rigorous testbed to drive the development exploration methods for RL, Right: An illustration of MATH-B' significant difficulty compared to common test sets like AIME24. Representative open-source models like Qwen2.5 achieve near-zero passe 0224 scores on MATH-B, highlighting its difficulty. Qwen2.5 results from Yue al. (2025).

---

Published as a conference paper at ICLR 2026
INTRODUCTION
In the 2010s, deep reinforcement learning showcased its power through striking demonstrations of exploration and skill acquisition (Mnih et al., 2013). Atari agents, starting from random play, mastered complex games by discovering strategies unreachable to base policies, guided by exploration incentives and intrinsic rewards (Ladosz et al., 2022; Amin et al., 2021). Methods such as count-based exploration (Bellemare et al., 2016) and later Go-Explore (Ecoffet et al., 2021) drove dramatic jumps from ieept play to expertise, highlighting RL's ability to uncover new capabilities. Around the same time, AlphaGo (Silver et al., 2016) and AlphaZero (Silver et al., 2017) extended this promise to board games like Go, Chess, and Shogi, where self-play took agents from scratch to superhuman mastery, revealing novel strategies along the way.
Against this backdrop, academic progress in bassonnn-focussee LLMs has taken a very different path. Community-trainee
models often show improved accuracy popular benchmarks such as MATH or AIME24 (Liu et al., 2025b; Song et al., 2025; Chen et al., 2025; Cheng et al., 2025; Cui et al., 2025; Shao et al., 2025; Wang et al., 2025; Yu et al., 2025b). However, these RI. models typically succeed only on problems that their corresponding base models could already solve given realistic sampling budgets (Wu et al., 2025; Yue et al., 2025). This is far cry from earlier RL successes, where base policies were incapable of solving tasks outright and progress required genuine exploration and skill acquisition. This disconnect between RL's exploratory promise and its current application reflects a substantial blindspot in the current open-source evaluation ecosystem. Because several open-source base models already achieve nearly 100% pass @ 024 on several popular benchmarks (Yue et al., 2025), test sets in their current form are fundamentally inadequate for measuring- or encouraging -eennine progress in reasoning beyond the base model's reach.
To address this gap, we introduce MATH-Beyond (MATH-B), a new benchmark of high- school-level competition math problems. specifically constructed so that popular open-weight base models are unlikely to solve even with 1024 attempts. As a result, progress on MATH-B necessarily requires expanding the reasoning capabilities of base models, making it an ideal target for academic research.
MATH-B is constructed by filtering mathematical reasoning datasets (DAPO-Math- 17K (Yu et al., 2025b) and DeepScaleR (Luo et al., 2025)), resulting in problems that are topically indistinguishable from those in standard benchmarks. While constructing the dataset, we also uncovered and addressed several non-obvious failure modes in programmatic verification, which informed our final benchmark design. To ensure correctness, all problems are additionally verified against stronger reasoning models such GPT-5-Mini
o4-mini: 04-iini-high, which reliably solve them. We further confirm that leading community RL models, including Nemotron-Research. Research-Reer -eae---eeaonnn-veeee DeepScaleR- 1.5B, perform poorly on MATH-B, underscoring the limitations of current approaches and the need for methods that extend reasoning capabilities. In summary, our contributions are as follows:
(Liu et al., 2025a) and
New benchmark suite: We construct MATH-Beyond, a benchmark suite derived from the failures of a large and diverse set of base models (pass@ 1 024 x 0). This suite includes: a union set of 181 problems unsolved by at least one of these models; model-specific benchmarks for targeted amalysis; and highly challenging intersection set of 41 problems that proved unsolvable for the entire considered set. To ensure quality, all problems are annotated for topic and difficulty following the procedure from Omni-MATH (Gao et al., 2024a), and undergo answer verification by frontier models (o4-mini-high & GPT5-Mini).
Verification pitfalls: During benchmark construction, we observed several pitfalls in standard RLVR verification. We take these into account in our benchmark design and also document them, highlighting subtle edge cases for the community to be aware of.
Model evaluation: We evaluate RL-finetuned models such Nemotron-Researche Reasoning-Qwen-L.555 DeepScaleR- 1.5B, and Skywork-OR1 (He et al., 2025a) on MATH- B and find that they do not substantially expand reasoning boundaries. In contrast, find that newer model families like Qwen3-4B and Qwen3-8B (Yang et al., 2025) perform better, presumably due better distributional overlap with our dataset.
2

---

Published as a conference paper at ICLR 2026
2 FRAMEWORK FOR EVALUATING MODEL EXPANSION
Our goal is to quantify whether a post-trained model (c.g., after RLVR) expands its reasoning capabilities beyond its base model. While post-training can affect many aspects (e.g., robustness, exploration, or transfer), our framework is designed to isolate the specific phenomenon of boundary expansion wwhtt new problems the post-trained model can solve relative its base model. This section formalizes this evaluation framework, adapting the nomenclature and definitions from Wu al. (2025) into a slightly simplified, empirical version. We apply this framework to our benchmark MATH-B, which is "zero-baveline" test. We define this as setting where benchmark is specifically constructed such that the base model empirically fails on all problems (i.c., its observed pass@k is zero) given realistic sampling budget.
2.1 THE EMPIRICAL P METRIC
We evaluate post-trained policy against its base model dataset D. Our evaluation is based on the empirical pass@k metric. For given problem x D and policy p E {лr 7) (usually an LLM), we draw k i.i.d. samples (v1...., UK). Let C(z) be the set of all correct completions for The empirical success is:
11 i (k, such that yı E C(x),
pass@k(p;a)
otherwise.
This binary value-1 for "pass" and 0 for "fail"-is the ground-truth measure of success for single problem.
With slight abuse of notation, the central metric reported in our paper is the average success rate across the entire dataset D:
passex(p)
passek(p;x),
D
xED
the ratio of problems in D solved by policy p using k-sample budget.
2.2 DECOMPOSITION AND KEY METRICS
To understand how T's performance differs from g's, we first define the Reachable Set R& (p. D) as the set of problems p solves:
Ru D) {ar D passēk(p; 1}.
Comparing the reachable sets of and allows to isolate primary metric and contextualize it.
Expansion (Primary Metric). Our primary focus and sole reported metric is the Expansion Rate. This measures genuine boundary expansion new problems solves that could not. It is defined based on the Expansion Set & = RR D) \ RRa (g. D):
E
Expansion Rate
D
Shrinkage. For diagnostic context, we also define Shrinkage (or "forgetting"), This measures problems q could solve that fails, defined by the Shrinkage Set S Rx(y, D) (T D). can be guantified as:
S
Shrinkage Rate
DD]
Preservation. Similarly, Preservation measures the fraction of g capabilities that retains, defined by the Preservation Set Pk = RR( D) n Re(g, D). It is quantified as:
|Pal
Preservation Rate = Relg, D)I
3

---

Published as a conference paper at ICLR 2026
Consolidation. Finally, Consolidation is concept for measuring if preserved solutions become more robust (i.e., solvable at pass@1), 1). It is defined as:
Ca q) = |Px n Rı (T,, D)I
IPAl
Interpretation. The overall pass rate of decomposes as pass@k(TT) -(Ex/++())/DDD While Shrinkage, Preservation, and Consolidation are crucial concepts for full diagmosis, our target for expanding the reasoning boundary squarely the Expansion Rate. The other concepts serve as theoretical guardrail to ensure that measured gains are from genuine expansion, not mere reallocation.
2.3 SPECIAL CASE: THE MATH-B BENCHMARK
The metrics framework Section 2.2 simplifies cleanly for MATH-B. This benchmark is constructed "zero-baseline" meaning it is composed of problems where the base model q was empirically observed to fail within the sampling budget (i.e. passêk x 0). In our evaluation, this means the base model's reachable set is empty::
Rx D) = Ø.
Under this premise, the decomposition from Section 2.2 collapses:
S Ø Pk Ø
& Rx( D).
Shrinkage and preservation are thus not applicable. Every solve by π is by definition an expansion, and the single relevant metric becomes:
Expansion Rate [R D)I pass@k(T).
D
A "win" on MATH-B is simply a positive Expansion Rate, providing an unambiguous readout of genuine boundary expansion.
CREATING THE BENCHMARK
3.1 ON THE PITFALLS VERIFICATION
The goal of our work is to identify questions that models fail at high pass@ 1 024 due to genuine methodological or calculation errors, not because of artifacts in the verification pipeline. Current rule-based verification methods for math, used both in training and evaluation, are themselves prone to systematic failures. These failures can mask true performance by penalizing answers where the reasoning or ground truth is technically correct but formatted poorly, making it appear as though the model failed when the verifier simply could not handle the formatting or parsing.
To guide our evaluation and benchmark design, and to alert the community to these pitfalls, we sketch seven distinct failure modes observed across common RL. finetuning and evaluation frameworks. We uncovered these issues through analysis of reasoning traces produced by PrepSeck-RI-Dissee Qwen2.5-7B (DeepSeek-AI) Al et al., 2025) when applied to a subset of NuminaMath- 1.5 (LI et al., 2024). Table provides concise but complete examples (problem, ground truth, model output snippet, and the precise mismatch), and Table in the Appendix summarizes how different frameworks are affected.
3.2 MATH-B CONSTRUCTION PIPELINE
Our benchmark is the result of multi-stage filtering pipeline designed to isolate problems that are (a) correct and unambiguous, (b) novel, and (c) demonstrably unsolvable by suite of open- base models even at high sampling budget. The process is detailed below and in Figure 1.
3.2.1 BASE DATASET SOURCING
Our construction begins by sourcing a large pool of candidate problems. The primary goal, as stated, to find problems that defeat open-source models high sampling budgets (pass@ @1 024). This
4

---

Published as a conference paper at ICLR 2026
Table 1: Common failure modes in rule-based math answer verification. These failures often stem from rigid heuristics, such as reading only the first or last boxed answer, requiring specific text anchors (e.g.. "Answer:"), or other parsing failures. Each row shows the ground truth (GT), a model snippet, and the resulting verifier error.
Mode
GT
Model snippet
Verifier behavior
F1: Multiple valid
or 13
13
Expected: accept hoth values. Actual: only first/last boxed read: one valid ignored.
F2: Late correct
150
50
100
150
Expected: final value only. Ac- sull: early intermediate cap- tured.
F3: Early correct
12
later
84
Expected: accept any correct boxed answer. Actual: later wrong overrides early correct. Expected: use corrected value. Actual: first match persists; "last-only" works, "first-only" fails.
F4: Corrections
25
20
then correcting 25
Unordered tuple
3, 4, 5
Expected: order-agnostic match. Actual: order-sensitive:
heuristics misgrade.
F6: Missing anchors
2 + 2 prefix)
1
(no "Answer:"
Expected: accept boxed nu- meral alone. Actual: anchor
quired: otherwise wrong.
F7: MCQ partial
CJ 1000
"The answer is C
Expected: accept label-only (C). Actual: requires exact "C)1000".
requirement immediately disqualifies many common datasets. For instance, DeepMath103K (He et al., 2025b) and Big-Math (Albalak et al., 2025) are explicitly designed to be solvable (c.g., for GRPO training), and indeed, models like Llama-3.1-88 solve them with few attempts. We also sought to minimize data contamination, ruling out datasets like NuminaMath which have been in pretraining by models like Qwen2.5-Math (Yang et al., 2024) and consequently DeepSeek-RI.
We therefore selected two base datasets: DAPO (Yu al., 2025a), which satisfies our criteria for both high difficulty (not verified by open-source models) and novelty (released after DeepsSeck-RR and DeepScaleR (Luo et al., 2025), which, while potentially seen, provides large corpus of problems that also lack open-source verification. This combined set forms our initial candidate pool of 53,682 problems.
3.2.2 QUALITY FILTERS
We then apply an array of quality filters as described in the following paragraphs.
Question and Answer-Type Filtering To mitigate the verification pitfalls detailed in Section 3.1, apply several deterministic filters. First, to avoid ambiguity from failure modes like F1 (Multiple valid answers), and F5 (Unordered Tuples), we filter DeepScaleR to include only problems with integer-based ground truths IDAPO-Math-173 already meets this criterion). Second, using regex- based filters, we remove all multiple-choice questions (to prevent F7 parsing failures) and any questions containing Chinese characters. Finally, to ensure all problems are self-contained, remove any questions referencing external figures or images. This filtering process reduced the pool 34,515 datapoints.
Pre-screening and Random Sampling The problems from the previous step still represent computationally prohibitively large set for lcrge-scale evaluation. To reduce this pool, we first perform difficulty pre-screening step. We evaluate Deepseek-R1 --ii----iiiilll---e2-----7777 all problems,
5

---

Published as a conference paper at ICLR 2026
keeping only those that it could not solve within pass@l 6 budget. In addition to the screening, randomly sample a portion of this dataset for further processing.
Ground- Truth Answer Verification A critical step is to ensure that these problems are unsolved due to their intrinsic challenge, not because their provided ground-truth answers are incorrect. While our source datasets are generally reliable (e.g., DeepScaleR is derived from AIME (MAA) and AMC (MAA, 2023) competitions (Luo et al., 2025)), we took an additional step to ensure the ground- truth answers are correct. We evaluate (pass@2) this pre-screened subset using two frontier-class models, 04-mini-high and GPT-5-Miní(Balunovi et al., 2025). For each problem, we prompt the models to "Think step-by-step and put your final answer \boxed. " and check if the extracted answer matches the dataset's ground truth. We retain only the problems where at least one of these frontier models successfully reproduced the ground-truth answer.
Deduplication Against Standard Benchmarks To ensure the novelty of our dataset, first perform an exact string-match deduplication of our candidate problems against several common test sets, including MATH-500 (Hendrycks al., 2021), MinervaMath (Lewkowycz al., 2022), OlympiadBench (He et al., 2024), AMC23 (MAA, 2023), AIME-2024, and AIME-2025 (MAA). We confirmed that no question from our set is present in these benchmarks. At the end of this stage, collection of 184 problems remain.
3.2.3 FINAL BENCHMARK CONSTRUCTION (P ASS 024 FILTERING)
The final step in our benchmark's ' construction is to filter the candidate problems using a diverse suite of representative open-source models. The definition of a 'base model' is contextual; it typically refers to model intended for further fine-tuning. Keeping in mind the models the community often uses for post-training research, we select a suite of models categorized as either 'base' or 'supplementary" models. As we will detail, these two groups are used to construct different subsets of our final benchmark.
Base Models This set is used to define the most challenging subset of our benchmark. It includes: Qwen2.5-1.5B, Qwen2.5-7B (Qwen et al., 2025), Qwen2.5- 1aae2.5-Math--aee 1.5B. Owen2.55Math- (Yang et al., 2024), Qwen3-4B-Batt -Base,, Owen3-8B-Base (Yang et al., 2025), DeepSeek-RR -i----Diiililleeee222 1.5B, DeepSeek- R1----R1--iiilliil -isiiiilll-Qwen225-77 (DeepSeek-Al et al., 2025), OLMo-7B (Groeneveld et al., 2024), OLMo--27B (OLMo al., 2025), and Llama-3.1-88 (Grattafiori et al., 2024).
Supplementary Models It includes: Awve22.5-15B-IInttruce Qwen22.577-Innttreet (Qwen al., 2025). Swen2.5-Math-1.5B Instruct, Qwen2.5-Math-77 Ma-atth---nnsrueee (Yang et al., 2024), Qwen3-4B, Qwen3-8B (Yang et al., 2025), DeepScaler- .5 (Luo et al., 2025), Cemotron-Researde Reathh-Resoonnn-ueee Qwen-1.5B(y and v2) (Liu et al., 2025a), and Skywork-OR1-7B (He et al., 2025a).
This group is combined with the base models to define the full benchmark.
The deduplicated and pre-screened candidate set then subjected to our final filtering stage. We evaluate the problems against all listed models generating 1024 samples for each. During this evaluation, we apply our robust verification logic (see Table for failure modes) to ensure we were measuring genuine reasoning failures. Our experiments and analyses required over 20 000 A100 GPU hours. Please refer to Appendix A.2 for more details on the evaluation parameters.
This comprehensive evaluation allows us define our final benchmark, MATH-B, which comprises three broad subsets for targeted analysis:
MATH-Beyond- Set (MATH-B-U): The full benchmark of 181 problems, containing any problem that at least one model from our entire suite (both base and post-trained) failed to solve within 1024 attempts.
Set (MATH-B-I): A more challenging core subset of 41 problems, that all of our considered base models failed to solve. This is a hard subset of MATH-B-U (see Table 5 for the QA pairs).
BonT---eyond-Interetttnin
Model-Specific Sets: For any given model in suite, this is the collection of all problems in the Union Set that failed to solve (see Table 4). These sets enable fine- -nnaginnned testing of RL. finetuned models derived from one of the evaluated base models.

---

Published as a conference paper at ICLR 2026
Overall, MATH-B datasets serve benchmarks of reasoning capability expansion (see Section 2).
3.3
DATA CHARACTERISTICS AND KEY REMARKS
Domain Distribution
Discrete Mattrmmticss
Data Source Distribution
Gecmetry
17.16 (e-81)
Namber Thwary
15.05 (x-20)
DeepScalef
14.45 (m10))
Algebra
DAPO-MeTH-17
Precaiculkss
2.0% (yrd)
Statstics
0.0% 0.0%)p11)
10
20
30
40
Percenaage Problems
Figure 2: Characteristics the MATH-B-U dataset. Left subplot shows the distribution of math domains. Right subplot show the distribution of source datasets.
Domain and Difficulty Annotation analyze the characteristics of the MATH-B dataset, we annotate each problem for its mathematical domain and buman-perceive difficulty. We adapt the procedure from Omni-MATH (Gao et al., 2024a), using GPT-5 to perform the labeling based on contrastive prompting strategy (see supplementary material for the full prompt). This method leverages labeled examples from various math datasets to contextualize and assign scores to new problems and has shown to be aligned well with human judgment (Gao al., 2024a). selection of annotated problems from MATH-B-I is provided in Table 5. We also plot the distribution of ground truth answers Figure 4.
Math- Nathh-Beyond-Uiio
Math- caee--Beoon--neteeee
--337
Medion -450
Median 5.00
Range 1.0-7.5
Range
Percentage Problems (%)
Percentage of Problems
(912)
7.2%
7.3% 7:30
7.3 2.32
(1-18)
85% (+12
4.%
2.% 2.09
33
(9-2)
2.4
17%
(o-7) 9p-1)
1-11
5
3
6
Difficulty Rating
Figure 3: Difficulty distribution. The left subplot shows the difficulty distribution of the MATH- Beyond- Union set, while the right subplot shows that of the MATH-Beye BFTTH-Beyon---o Beeonnd--neessectioon set. The wide spread of difficulty levels highlights a key mismatch: the problems that models find challenging not necessarily those that humans typically struggle with.
Distribution Analysis As shown in Figure 2, the topic distribution of MATH-B consists entirely of standard high-school mathematics subjects, ensuring topical relevance. The difficulty distribution, plotted in Figure 3, reveals wide range for both the Union and Intersection sets, with median numann-dificuul rating of 4 out of 10. Notably, even for the challenging MATH-B Intersection set, the maximum difficulty score is only 6.5. This suggests significant disconnect between human-
7

---

Published as a conference paper at ICLR 2026
perceived difficulty and model failure modes; problems that not considered exceptionally hard for humans can still be robustly unsolvable by current models.
Benchmark Justification strong benchmark should be realistic, difficult, and efficient (Nie et al., 2025). MATH-B meets these with (i) realism: problems drawn from standard high-school curricula; (ii) difficulty: base models perform poorly even with a large sampling budget (i.e. pass 4); and (iii) efficiency: the small suite enables potentially rapid, low-cost evaluation, and-as as shown in Section 4-post-trained models exhibit only marginal gains at pass@ 1024, making MATH-B a perfect test bed for research.
Benchmark Usage We expect researchers to use MATH-B B with the base models and their cor- responding splits specified in Section 3.2.3: (1) evaluate the base model to estimate pass for reasonably large k (e.g., A = 1024, expected to be nearly O); (2) apply some RL method of interest; (3) re-estimate pass@k for the post-trained policy, which indicates the Expansion Rate per Section 2. The benchmark is intended specifically for methods that aim to expand the listed base model's reasoning boundary. While this is a research benchmark, we hope the methods that people come up with scales larger scales and other scenarios
EVALUATING EXPANSION ACROSS FINETUNING METHODS
We evaluate several post-trained models on MATH-B to measure their ability to expand beyond their base model's 's reasoning capabilities (i.e. Expansion Rate Section 2). Our analysis of mod- els finetuned with reinforcement learning (RI.) reveals that current methods achieve only modest expansion. As shown in Table 2, the three RI models based on DeepSeek-R11- --Dissiil-litttieeeenn 1,5B (rl-1.5b) (DeepSeek-Al et al., 2025) solve fewer than 10% of the test problems. In contrast, Skywork- OR1-7B (skywork_or __rl) (He et al., 2025a) reaches more promising 21% expansion; notably, its training involves adaptive entropy control and a higher temperature, likely affording a greater scope for exploration. Keep in mind that these passe 1 024 evaluations. This result suggests that RL techniques designed to explicitly encourage exploration may indeed result higher Expansion Rates. We also observe that while prolonged training (i.e. more RL compute) can offer marginal gains-as seen in the slight improvement from Nemotron-Researche Reathh-nnesoonnnneeeeeee Qwen--..nnn ver- sion (nemotron v1) et al., 2025a) the rather small 1.5% increase underscores the need for more efficient and effective exploration methods. See also the evolution of Expansion Rate in Figure 5.
Cemotron-Rescarce Research-Reasoning Reasoning- Qwen- 1.5B version 2 (nemotron v2) (Liu
Table 2: Expansion Rates of post-trained models using either RL or SFT/Distillation. The Expansion Rate measures the percentage of previously unsolvable problems (from the base model's perspective) that the post-trained model can now solve We additionally add AIME24 (pass @ I) numbers of the post-trained models to illustrate the difficulty of our dataset (He al., 2025a; Yang et al., 2025; Liu et al., 2025a).
Base unsolved
Expansion Rate (%, pass @ 1024)
AIME24 (pass @ I)
Base model
Post-trained
Method
Reinforcement Learning (RL) models
r1-1.5b
nemotron_v
RL
115
7.83
48.13
r1-1.5b
nemotron_ v2 RL
115
9.57
49.58
r1-1.5b
DeepScaleR
RL
115
5.22
40.21
r1-7b
skywork_orl RL
99
21.2
70.2
Supervised Fine-Tuning (SFT) models
Qwen3-4B-base
Qwen3-4B
Long CoT Dist.
112
58.93
73.3
Qwen3-8B-bar
Qwen3-8B
Long CoT Dist.
66.38
76.0
As an illustrative contrast, we evaluated Qwen3-4B and Qwen3-8B. These models are the result of finetuning their respective base versions (Qwen3-4B-Bi) 4B-Base and Qwen3-88 (ron3-BB-BBl 8B-Base) by distilling long Chain-of-T Mhain-of-Though Thought (CoT) reasoning trajectories from more capable teacher model. They demonstrate substantially higher Expansion Rates of 58, 8893% and 66.38%, respectively. While this is not direct topples-o-apples comparison due differing training setups, the result is highly informative: it
8

---

Published as a conference paper at ICLR 2026
shows that significant expansion is achievable when model is exposed to the correct distribution of reasoning steps, an overlap that long CoT distillation provides (Yang et al., 2025).
This contrast highlights that the primary limitation of current RL techniques may not be inherent inability of the base models to learn, but rather the failure of the exploration process to find these effective reasoning pathways its own. Developing RL methods that can discover these pathways without a teacher model remains a key challenge for reaching frontier capabilities and is the central motivation for our work.
DISCUSSION AND RELATED WORK
On the Choice of k = 1024 Our selection of l: = 1024 for all passe k evaluations is deliberate choice to ensure our benchmark is challenging, stable, and efficient. Firstly, large sampling budget is necessary to probe the true reasoning boundary of model. Many popular benchmarks become saturated at this scale, with base models solving a high percentage of problems and leaving no room to measure improvement (Yue et al., 2025). We chose where MATH-B remains difficult, creating a meaningful testbed for genuine expansion. Secondly, this choice is strongly supported by our empirical analysis. While overall pass@k performance shows a consistent log-linear increase with the sampling budget (Figure 6), the marginal gains for each additional sample diminish considerably (Figure 7). Most importantly, the Expansion Rate for RL-finetuned models-our core metric for progress-laarely plateaus as the budget approaches 1024 (Figure 5). Therefore, beyond their comfort zone on challenging benchmark, yet provides a stable and computationally feasible point to reliably measure the expansion of the reasoning boundary.
1024 precisely because it represents budget
1024 represents principled trade-off: is large enough to push models
Limitations of Existing Benchmarks and Metries In mathematical reasoning, progress is often measured by pass@k. For k > 1, this metric is taken to indicative of a model's exploratory potential. However, pass is an incomplete measure of exploratory potential, as it conflates the sharpening of existing solutions (Consolidation) with the discovery of entirely new ones (Expan- sion) (Wu et al., 2025). As detailed in Section 2, work is concerned with Expansion. Many existing benchmarks are now saturated by strong open-source base models, making it impossible to measure new boundary expansion (Yue et al., 2025; Balunović al., 2025; Wu et al., 2025). Furthermore, these benchmarks are targets of hyper-optimization, potentially rewarding spurious correlations (Shao al., 2025). MATH-B addresses this by serving as a diagnostic tool. It comprises problems that are topically standard high-school math but are constructed to expose the subtle failures and brittleness of the dominant open-source research paradigm.
Our Empirical Contribution in Context Building on the framework of Wu et al. (2025), we instantiate these ideas in a concrete, reusable benchmark. their work introduces and analyzes Expansion within a specific setting, whereas we put forth a zero-baseline dataset (i.e., pass@ @1 024 x 0 for the base models) that operationalizes the concept and scales evaluation across a wide range of open-weight models. This offers practical path to shift community focus from merely improving pass@k 0k on saturated suites to demonstrably expanding the reasoning boundary. Further, while MATH-B is instantiated for specific set of open- weight base models, we expect subset of items to remain zero- bare-baslliee for larger open-weight models. We did not evaluate those due to resource constraints. Since MATH-B is intended drive RL methods that expand a given base model's s reasoning boundary (rather than chase model-specific quirks), we expect the resulting methods and insights transfer across model families and scales.
CONCLUSION
Our findings indicate that current post-training methods in the open-source ecosystem, particularly for models up to 8B parameters, primarily refine pre-existing reasoning abilities rather than creating new ones. The poor performance of these post-trained models on MATH-B empirically confirms that they struggle to expand their capabilities to problems that lie just beyond their base model's reach. We introduce MATH-B B as a precise diagnostic tool to address this issue. Its purpose is to catalyze research into post-training methods that achieve genuine exploration, providing a clear and reliable signal for when a model has truly expanded the boundaries of machine reasoning.

---

Published as a conference paper at ICLR 2026
7 REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our work. All code required to reproduce the experiments provided in the supplementary material. Detailed derivations are included in the main text (Section 2). Experimental settings and hyperparameters are described in the main paper (Section 3.2) and supplementary sections (Appendix A.2).
REFERENCES
Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, and Nick Haber. Big-math: A large- scale, high-quality math dataset for reinforcement learning in language models, 2025. URL
https: / wttpaaaiiv.ggaaaaaaaaaaaa/aaaaaa /2rpg//bb/2502.. 250.211377
Susan Amin, Maziar Gomrokchi, Harsh Satija, Herke Van Hoof, and Doina Precup. A survey of exploration methods in reinforcement learning. arXiv preprini arXiv: (tt.::200000057... 2021.
Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. Math- arena: Evaluating ilms on uncontaminated math competitions, February 2025. URL https:
/ h/matharena.aat ai/.
Mare Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems, 29, 2016.
Zhipeng Chen, Xiaobo Qin, Youbin Wu, Yue Ling, Qinghao Ye, Wayne Xin Zhao, and Guang Shi. Pass @ k training for adaptively balancing exploration and exploitation of large reasoning models. arXiv preprint 2508v:2508. 10751, 2025.
Daixuan Cheng, Shaohan Huang, Xuckai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. Reasoning with exploration: An entropy perspective. arXiv preprint arXiv:2506.14770 2025.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22611 2025.
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. 2025. URL https: http://axxxw.oo.a/////////////// 12948.
Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley, and Jeff Clune. Go-explore: a new approach for hard-exploration problems, 2021. URL https: arxiv. wwtpxvivvrrr...oo 10995.
org abs abs//ab1.111 1900.
Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni- math: A universal olympiad level mathematic benchmark for large language models, 2024a. URL https: / arxiv org/ alss 2410 0 07985.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman. Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muennighoff, et al. The language model evaluation harness, 07 2024b. URL https: / /ww//ezodooo//oo org/ records/ 1266860..
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, al. The llma herd of models, 2024. URL https: 21783.
a/ /aaxiivoorr abs
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, et al. Olmo: Accelerating the science of language models, 2024. URL https: : /ttpxixxii///////////e/////aall/ owg/abs/ 24422.08333
10

---

Published as a conference paper at ICLR 2026
Nathan Habib, Clémentine Fourrier, Hynek Kydlíček, Thomas Wolf, and Lewis Tunstall. Lighte- val: A lightweight framework for Ilm evaluation, 2023. URL https://github com/
thugiiggacce/lighin eiaallval.
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi. Zhiyuan Liu, and Maosong Sun. Olympiad- bench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024. URL https: / www.v.oog.g.....e..e.oo.oo.o.ooe
orgg abs/////////// abwp////...... 2402 14008.
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. Skywork open reasoner 1 technical report, 2025a. URL https: //arxiv. org/ abs 2505. 22312.
Zhiwei He, Tian Liang. Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath-103k. A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning, 2025b. URL https: arwiv. org/ abs 2504.0000 11456.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Mcasuring mathematical problem solving with the math dataset, 2021. URL
https: /ttpaaaivvv.///////e///aaa org/ a 03877.
Hynek Kydliček. Math-verify: Math verification library, 2024. WtRLhttss:///tthtee huggiingfacee/mth-vveii
Pawel Ladosz, Lilian Weng, Minwoo Kim, and Hyondong Oh. Exploration in deep reinforcement
learning: A survey. Information Fusion, 85:1-22, September 2022. ISSN 1566-2535. doi: 10.1016 jinffus.2022.03.000. URL http:://ee/e //x. doi. org.orgg////o 10. 101 j/j. f.f.uus.222222 2022. 03.003.
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, 2022. URL https:tp//.. /ttpaviivvv/..////////.////a.ava wwg. abs 220 6.02206.000 14858.
Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi. Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. (https://oe /tte::///tttttt wwwpppooocttttnoiioooa http:///eepottttunnaaaaaaaaeette
hottps://huggi... httph/hggggggagaaoooooooocooc face..
NTTT-MMNNNmiiaaaaa aimo aimoo-progress-piiieeeeeae
2024.
Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models, 2025a. URL httph::t//// //axvvv.///ii/////a/////a/ org/ abs/2505 24864.
Zichen Liu, Changyu Chen, Wenjun Li. Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025b.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol ol-preveew with a 1 1.5b model by scaling rl. https: pettss:/tingen /ttintiinurrrrrr//ee/errerreeele wtw.cooo/////// KD2347, 2025. Notion Blog.
MAA. American invitational mathematics examination aime. In American Invitational Mathematies Examination AIME.
MAA. American mathematics competitions. American Muthematics Competitions, 2023.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, loannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, 2013. URL
wttpa//wwwww.////////11.1. https////arww..rrraa///////// 5602.
11

---

Published as a conference paper at ICLR 2026
Fan Nie, Ken Ziyu Liu, Zihao Wang. Rui Sun, Wei Liu, Weijia Shi, Huaxiu Yao, Linjun Zhang, Andrew Y. Ng, James Zou, Sanmi Koyejo, Yejin Choi, Percy Liang, and Niklas Muennighoff. Uq: Assessing language models unsolved questions, 2025. URL 2508. 17580.
abs/
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, et al. 2 olmo? 2 furious, 2025. http::::: //axivv..//iii/////a///i/ar orgg abaaabbs/2500 00656.
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, et al. Qwen2.5 technical report, 2025. URL https: :/sp::://aaa :ttp://aaavvv/..//ea/eeetaaaa. ott.aaiwwi.aaa.aaaaaoooaaaooooat wttpg/////////////////////saase/ //t.//411..11111.......
Negin Raoof, Etash Kumar Guha, Ryan Marten, Jean Mereat, Eric Frankel, Sedrick Keh, Hritik Bansal, Georgios Smyrnis, Marianna Nezhurina, Trung Vu, Zayne Rea Sprague, and Ludwig others. Evalchemy, 2025.
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang. Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, et al. Spurious rewards: Rethinking training signals in rlvr. arXiv preprint urXiv:2506. 10947, 2025.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024.
David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonogiou, Veda Panncershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of go with deep neural networks and tree search. Nature, 529:484 403,, 2016. URL http: //www.nature....mmmmmm...m
nature. com wwwm/aature/oouraaa//...aaaaaate /ttp:///////ntt//feaa//aaeteelle
fwll//llll/////ll //wattaaaaeeeeeeeeeeeeeeeeeeeeee 6961 html.
David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Si- monyan, and Demis Hassabis. Mastering chess and shogi by self-play with a general reinforcement learning algorithm, 2017. URL https: arxiv. wwt.///vooooooooooooooooooooo abs/ 11/1111..000 01815.
Yuda Song, Julia Kempe, and Remi Munos. Outcome-based exploration for Ilm reasoning. arXiv preprint arXiv:2509.0699 2025.
Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforcement learning, 2024. URL https: h////itthuuuu com/ httm///ggginggaa//tt...
Yiping Wang. Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, al. Reinforcement learning for reasoning in large language models with one training example. arXiv preprini arXiv:2504.20571, 2025.
Fang WU, Weihao Xuan, Ximing Lu, Zaid Harchaoui, and Yejin Choi. The invisible leash: Why rlvr may not escape its origin, 2025. URL https:httttteeeeeeeeeee .rgg abs//bb/50000 14843.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5 technical report: Toward mathematical expert model via self-improvement, 2024. URL https: peatttss://rrees //axivi v. org/ / abs / 244 24.9.12122 12122.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, et al. Qwen3 technical report, 2025. URL https: /ttpss///axvvv.// arwiv.iiivvv///iv orge abs/ 2505.09388
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin. Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangiie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song. Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source Ilm reinforcement learning system at scale, 2025a. URL https: / arxiv.org// abs/2503. 1 14476.
12

---

Published as a conference paper at ICLR 2026
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476. 2025b.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model?, 2025. URL https: / arxiv, wwtg/ab////0//eo/
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 VERIFICATION ISSUES IN DIFFERENT FRAMEWORKS
Table 3: Verification failure mode vulnerabilities across frameworks. V= vulnerable, - -esilli, resilient,
VI- partial. Frameworks: TRL (Transformers RL) (von Werra et al., 2024), VERL (Sheng et al., 2024), LM-Eval (hendrycks/minerva) (Gao et al., 2024b), LightEval (Habib et al., 2023), SCORE (LM-Eval SCORE math) (Gao et al., 2024b), evalchemy (ZeroEval) (Raoof et al., 2025). HMMT (evalchemy matharena) (Raoof et al., 2025), Math-V (Math- (Math-Verifyy) (Kydličck, 2024). Methods: Firstit = first number string; Right= rightmost priority; Ltd. = limited support (55 expressions).
FI-+7: See Table for detailed descriptions.
Failure Mode
TRL
VERL
LightEval
LM-Eval
SCORE
evalchemy
HMMT
Math-V
Fl: Multiple solutions (OR)
V
V
V
51
VT
F2:
Late correct
41-
11-
F3: Early correct
1
F4: Answer corrections
V
4
4
4
Unordered sets
V
/
F6: Missing anchors
V
F7: MCQ partial match
V
V
V
V
V
V
Method
First
Last
Right
Last
First#
Last/All
Right
Anchors
Yes
No
No
No
No
No
No
Yes
Multi-Answer
No
No.
Ltd.
No.
Partial
No
Yes
Ltd.
A.2 INFERENCE PARAMETERS FOR EVALUATION
All models were run with nucleus sampling at top_p of 0.95. Other parameters were according
to model-specific recommendations:
OLMo Models:
Temperature of 1.0, with max max_maaa_eeeeaaeeea tokens set 2048 (vl) and 4096 (v2).
Qwen2.5-Math Models: Temperature of 0.6 and max tokens of 4096.
All Other Models:
Temperature of 0.6 and max max_tokens of 32,768.
A.3 MATH-B DETAILS
In Figure 4, we show the distribution of final answers for MATH-B-U, which spans broad range of integer values.
A.3.1 LOOKING AT SOME SAMPLES FROM MATH-B
In Tables 5 and 6, we list ten randomly sampled MATH-B-I questions-the base split and the full set, respectively.
A.4 ANALYSIS PASS SOK PERFORMANCE SCALING
To further justify our evaluation methodology, we analyze the performance scaling of all 21 models from Table on the MATH-B Union set.
In Figure 6, we plot the complete passšk evolution the sampling budget k increases up to 1024. The performance curves for nearly all models exhibit characteristic log-linear growth, indicating that improvement is consistent but not linear with computational effort. While this trend suggests continued gains with more sampling, it also shows initial signs plateauing at higher values of
To better quantify this observation, Figure 7 visualizes the marginal gain in performance. Specifically, it plots the average increase in the pass@k rate for each successive 64-sample increment. This plot clearly illustrates the principle of diminishing returns: the most significant gains concentrated lower sampling budgets, and the rate of improvement slows considerably as the budget approaches 1024. Together, these figures provide strong empirical support for our choice of 1024
14

---

Published as a conference paper at ICLR 2026
Distribution of Final Answer Values (n=181)
1012
Total:
MAX 1,000,000,000,000
10"
1,000,030,000 000,004
Mear: 11,0516678,08 5
10
408.0
107
10a
Answer (symlog
10
10
2.440
PSI
10'
P2S 00
102
-10
MINN -3
-10
25
50
75
100
125
150
178
Problem (sorted by answer value)
Figure 4: Distribution of ground truth (final-answers) MATH-B-U, We use the log-scale for better readability.
practical and stable point for evaluation, beyond which brute-force sampling becomes an increasingly inefficient path solving the remaining hard problems.
25
Expansion (%)
15-
DoepScatoft
64
128
255 Passšik (log scale)
512
1024
Figure 5: Evolution of Expansion Rate for RL Models. Models evaluated the MATH-B problems failed by their respective base models (115 for R1-QWenn.5-115BB 99 for R1-Qwen2.5-7B).
A.5 LLM USAGE
The authors this submission use IDEs with built-in LLM support, so LLMs have been used to help with menial coding tasks. Further, we used models like GPT-5, Gemini for rephrasing several
15

---

Published as a conference paper at ICLR 2026
Table Models" number of unsolved questions at pass@1024. Models are grouped into Base vs. Supplementary. "Intersection (base)" and "Intersection (all)" indicate the overlap of unsolved problems across models.
Model
Unsolved
Type
Notes
Qwen2.5-1.35
145
Base
Qwen2.5-7B
120
Base
Owen2.5-Math-1L..
115
Basc
Qwen2.5-Math-7B
101
Base
Qwen3-4B-Base
112
Base
Qwen3-8B-Bam
116
Base
DeepSeek-R1
115
Base
Distill
DeepSeck-RI-On
99
Base
Distill
OLMo-7B
158
Base
OLMo-2-7B
132
Base
Llama-3.1
151
Base
Owen2.5-1.5B-Instrue
124
Supplementary
Swee2.5-7B-Instruu
131
Supplementary
Owen2.5-Math- Instruct
139
Supplementary
Qwen2.5-Mathe -nnstruttt
117
Supplementary
Qwen3-4B
67
Supplementary
Qwen3-8B
Supplementary
DeepScaler-1.55
142
Supplementary
Nemotron-1.5B..
137
Supplementary
Momooron-1.5BB-v
142
Supplementary
Skywork-OR1-7B
103
Supplementary
Infersection (base)
41
Shared failures across all base models Shared failures across all models
Intersection (all)
13
Models: 1-48 Parmesters
Model:: 7:40 Peramebers
Paazok saaa)
Passi scale)
24
Figure 6: Evolution of pass@k for all models on MATH-B-U.
paragraphs in this manuscript. In addition to that, we used GPT-5-mini and 04-mini-high to label the difficulty and topics of the benchmark we create (Section 3.2).
16

---

Published as a conference paper at ICLR 2026
Table 5: Question texts (verbatim) from MATH-B-I (base). Each question includes the sentence "Let's s think step by step and output the final answer within \boxed appended.
ID
Question (wrapped to fit)
Difficulty
Final Answer
Fisica and Ritmo discovered piece of Notalium shaped like rectan- gular box, and wanted find its volume. To do so, Fisica measured its three dimensions using ruler with infinite precision, multiplied the results and rounded the product to the nearest cubic centimeter, getting result of 2017 cubic centimeters. Ritmo, on the other hand, measured each dimension the nearest centimeter and multiplied the rounded measurements, getting a result of cubic centimeters. Find positive difference between the least and greatest possible positive values for V.
4.5
7174
2 Let N denote the natural numbers. Compute the number of functions : + {0, 1..... 16) such that f(x + 17) f(x) and f(x²)
6.0
12066
f(x)² 15 (mod 17) for all integers
1.
3
distinet letters in the following subtraction problem repre- sents a different digit. Find the number represented by the word TEAM. PURPLE COMET TEAM.
Each
6852
4 Find the of all positive integers such that there exists integer the same the
4.0
1026
with [6] * such that the base representation base representation of n.
5
Let S be set of size 11. A random 12-tuple (s1...., .12) of elements of S is chosen uniformly random. Morcover, let permutation
6.0
1000000000004
- S be chosen uniformly at random. The probability that 8141 & π (8:) for all < S 12 (where $13 = 81) can be written as where a and are relatively prime positive integers. Compute a.
6 Let N denote the natural numbers. Compute the number of functions N {0. 1..... 16) such that f(x + 17) f(x) and f(x²)
4.5
12066
f(x)² + 15 (mod 17) for all integers x
7
A unicorn is tethered by a 20-foot rope to the base a cylindrical tower of radius feet. The rope is attached to the tower at ground level and to the unicom at height of 4 feet. The rope is taut, its end is 4 feet from nearest point on the tower, and the length rope touching the tower a-vè feet, where a, b, c positive integers and is prime. Find a c.
3.5
813
8 For which maximal N does there exist an N --digit number such that among any sequence of consecutive decimal digits some digit is present only once?
6.0
1023
9
P= {(2, y) < æ. S 25, æ, E Z). Let T the of trian- gles formed by picking three distinct points in P (rotations, reflections. and translations count distinct). Compute the number of triangles in T with area larger than 300.
436
that 5-by-5 grid square pieces, she only wants to cat the center piece. She repeatedly (i) chooses gridline and splits the bar, (ii) discards the part not containing the center, (iii) repeats until only the center piece remains. Compute the number of possible sequences of operations.
Katie has a chocolate
3.0
6384
17

---

Published as a conference paper at ICLR 2026
Table 6: Question texts (verbatim) from MATH-B-I (all) set of all considered models in Sec- tion 3.2.3, Each question includes the sentence "Let's think step by step and output the final answer within \boxed appended.
ID Question (wrapped to fit)
Difficulty
Final Answer
Bob is writing sequence of letters of the alphabet, each of which can be either uppercase or lowercase, according to the following two rules: If he had just written uppercase letter, he either write the same letter in lowercase after it, or the next letter he had just written a lowercase letter, he can either write the same letter in uppercase after it, or the preceding letter of the alphabet in lowercase. For instance, such sequence is aAaABCI Ddc How many sequences of letters he write that start at (lowercase) and end at (lowercase) x?
1.5
376
alphahet in uppercase. If
Let V = fl,..., 8). How many permutations V + V are auto- 2
5.5
30212
morphisms of some tree? (A graph consists of some set of vertices and some edges between pairs of distinct vertices. is connected if every two vertices in are connected by some path of one more edges. A tree G on V a connected graph with vertex set V and exactly [V] edges, and an automorphism of G' is permutation vertices j. V connected by edge iff (i) and arj) are.)
- V such that
Find sum of all positive integers n such that there exists an integer with [6] + 4 such that the base - represetattion of n is the same as the base b representation of n.
1026
4
A unicorn is tethered by 20-foot silver rope the base of magician's cylindrical tower whose radius is feet. The rope attached the tower at ground level and to unicorn at height of feet. rope taut, the end of the rope is 4 feet from the nearest point on the tower, and the length of the rope touching the tower is e-vh feet, where a, b, c are positive integers and is prime. Find b
3.5
813
5
16x16 square sheet of paper is folded once in half horizontally and once half vertically make an square. This square again folded in half twice to make a 4x4 square. This square is folded in half twice to make 2x2 square. This square is folded in half twice to make 1x1 square. Finally, scissor is used to make cuts through hoth diagonals of all the layers of the 1x1 square. How many pieces paper result?
6.0
5414
Mau writes permutation of (1, 2, 3....... 103 across his paper with the leftmost number equal to and the rightmost equal to 10. Exactly one interior number (not including or 10) is less than both its immediate left and right neighbors. How many such permutations
1636
there?
denote the natural numbers. Compute the number of functions ., 16) such that f(r + 17) = f(x) and f(x²) = + (mod 17) for all integers x 1.
12066
M 14 - (0, 1........
8
Sir Alex plays the following game are empty. each move either (1) chooses any number of the form 2 (nonnegative j) and puts into an empty cell, or (2) chooses two cells with the same number 2', replaces the number in one cell with 25-1 and erases the number in the other cell. At the end, one cell contains 2 and the others empty. Determine the maximum number of moves possible terms of n. Provide the value when v 10.
of cells. Initially all cells
5.0
2025
9
Alice and Bob play on board one row of 2022 consecutive squares. They take turns placing domino tiles that cover two adjacent squares; Alice goes first. A tile must not game ends when no can placed. Alice wants maximize the numher uncovered squares at end; Boh wants to minimize What is the greatest numher of uncovered squares Alice can ensure, no matter how Boh plays?
4,0
290
square already covered. The
10 (Caos)
[sic] has 6 legs, each side. A walking pattern is ordered sequence raising and lowering each the legs exactly once (total 12 actions), starting and ending with all legs on the ground. The pattern is safe if at any point he has at least 3 legs the ground and not all three legs the same side. Estimate N, the number of safe patterns.
4.0
1416528
18

---

Published as a conference paper at ICLR 2026
Average Marginal Gain (N)
141
0.8
6.7
M-120
13b-2
4db-522
322-1
575
Passgk Incroment
Figure 7: Average gains in pass@k relative to the size of MATH-B-U Averaged over 21 models, the rate solving new problems per 64-sample increment decreases as the total budget grows, demonstrating diminishing returns.
19


Reviews:
[{"review_id": "A4I0CAXgVl", "reviewer": "Reviewer_jrWD", "summary": "The paper introduces MATH-Beyond (MATH-B), a set of small mathematical benchmarks designed to exclusively measure the Expansion Rate of post-trained LLMs, which is the percentage of problems that post-trained models can solve but base models do not even at high sampling budget (1024). The central motivation is to address the debate that current RL methods primarily sharpen existing base model skills rather than discover entirely new reasoning capabilities (Expansion).", "questions": "Please see my weaknesses section.\n\nCould the author comment on the expansion rate of the 2 deepseek-r1-distill models, since they are finetuned from a teacher model as well? This is related to your paragraph lines 406 - 411. Does it make sense to include them in the base model list?\n\nI don't understand the division into base and supplementary models and what it is supposed to show.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is a meaningful contribution to the community debate regarding whether RL-trained models truly discover new reasoning skills. These benchmark can provide good signal on this, as well as how to improve current methods.\n\nThe paper clearly defines the metric and is very detailed on their data filtering pipeline which makes readers more confident about the correctness of their benchmarks.", "weaknesses": "The resulting datasets are small enough that the authors could possibly annotate themselves, and perhaps provide further qualitative insights on these questions. In particular, why are these problems hard for the models but their GPT5-rated difficulties are lower (which the author calls  human-perceived difficulty).\n\nI believe there are questions in the pre-filtered dataset that are rated as more difficulty (see the Omni-MATH paper, which dataset is also used here), yet somehow they end up solvable by some models, which confuses me. Could it be evidence of data leakage?\n\nThe number of benchmarked models is quite small, only 4 pairs, and all models are smaller than 8B. How do the benchmarks hold up for larger models? This would enhance the benchmark's longevity.\n\nThe authors should discuss the RL post-training data used for the post-trained models in section 4. Is there train-test leak? For example, in the pair r1-1.5b + DeepScaleR, does the training data contain some same problems as in your proposed test datasets? Similarly, the authors should discuss the SFT training data.\n\nI'm willing to raise my score once the weaknesses are addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7VFUduPo6e", "reviewer": "Reviewer_uHA5", "summary": "This paper introduces MATH-Beyond (MATH-B), a diagnostic benchmark designed to evaluate whether post-training methods genuinely expand mathematical reasoning capabilities or merely refine existing ones. Observing that base models with large sampling budgets (pass@1024) already solve nearly all problems on widely used benchmarks, the authors construct MATH-B to challenge open-source models up to 8B parameters even under large sampling budgets. MATH-B serves as a \"zero-baseline\" diagnostic tool to catalyze research into exploration-driven RL approaches that achieve genuine capability expansion.", "questions": "1. How do you ensure that MATH-B problems require genuine reasoning exploration rather than reflecting the specific failure modes of the filtering models? \n2. What are the results on large models (>8B), and does the benchmark maintain discriminative power at scale?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper addresses an important problem in post-training research, evaluating whether models genuinely explore new reasoning paths versus merely optimizing existing solution modes. This distinction between capability expansion and capability refinement is crucial for understanding the true progress in mathematical reasoning.\n2. This presents a structured approach to dataset construction using automated methods, which could potentially improve efficiency compared to pure manual annotation.\n3. The paper provides extensive experimental validation across multiple RL fine-tuned models and large sampling budgets to demonstrate the benchmark's effectiveness. The empirical results convincingly show that existing post-training methods struggle on MATH-B, validating the benchmark's design as a diagnostic tool.\n4. The paper is well-written with clear motivation and logical flow.", "weaknesses": "1. This paper uses models with <8B parameters for data filtering may introduce model-specific biases. The filtering model's capability ceiling and training data preferences could propagate into the dataset, potentially affecting the benchmark's reliability and fairness across different model architectures and training paradigms.\n2. The paper only evaluates models ≤8B parameters, leaving the benchmark's effectiveness for larger models (>8B, especially >70B) unverified. Without validation on state-of-the-art large models, it remains unclear whether MATH-B maintains discriminative power or exhibits ceiling effects for models that mainstream research focuses on, potentially limiting the benchmark's practical utility.\n3. Evaluating the same or similar models that were used for data filtering may introduce circular dependencies and bias.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Kw6gwWjT15", "reviewer": "Reviewer_VmSx", "summary": "1.Improving performance is explored on the proposed benchmark via RL requires methods that learn to reason in ways that go beyond base model capabilities in repeated sampling. Since the problems are drawn from subsets of DAPO-Math-17K and DeepScaleR datasets, they remain topically equivalent to standard high-school math.", "questions": "1.Why does the proposed dataset fail to demonstrate performance gains when used for reinforcement learning (RL) training?\n2.Why are difficulty levels and other metadata flags not provided for datasets such as AIME24 and AIME25?\n3.Why is the quality of the proposed dataset, generated via an LLM-based algorithm, not validated through human expert review?\n4.Why is there no evidence showing that a diverse set of LLMs improve their reasoning abilities when trained on the proposed dataset?\n5.Why is there no comparative analysis between the proposed dataset and established reasoning math datasets such as MATH-500, AIME24, and AIME25?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The dataset is hard for 1.5b llm to follow, RL fine-tuned models such as Nemotron-Research-Reasoning-Qwen-1.5B and DeepScaleR-1.5B-Preview perform poorly on MATH-B at pass@1024.", "weaknesses": "1.The proposed dataset does not show the goodness on the performance rising after using the dataset for RL training.\n2.The difficulity and other flag is not given for the dataset such as AIME24 and AIME25.\n3.The quality of the proposed dataset from the LLM-based algorithm is not double-check on human experts.\n4.These is no proof to show a variety of LLMs improve their reasoning ability on the application of the proposed dataset.\n5.There is no comparison with the proposed dataset and other famous reasoning math dataset such as MATH-0500, AIME24, AIME25.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B6fIiI6mrJ", "reviewer": "Reviewer_aWPb", "summary": "This paper introduces MATH-B, a benchmark designed to evaluate whether reasoning models fine-tuned with RL can genuinely “expand” beyond the capabilities of their base models, aiming to push progress past the current plateau. The benchmark is constructed by filtering existing datasets, DAPO-Math-17K and DeepScaleR, to retain problems unsolved by several open-source models even under a large sampling budget (pass@1024). The authors define an “Expansion Rate” metric, effectively pass@k on a zero-baseline set, and evaluate several RL-trained and SFT models. They find that most RL methods yield only modest (<10%) improvements, whereas long-chain-of-thought (CoT) distillation achieves substantially higher (>50%) gains. The authors argue that MATH-B can serve as a diagnostic tool to guide future exploration-driven RL research.", "questions": "1. In Figure 3, the authors show that the problems models find challenging are not necessarily those that humans typically struggle with. Why does this mismatch occur, and what specific failure modes of LLMs lead to zero-success rates across multiple models?\n\n2. For problems outside MATH-B that models solve with very low probability, is success mainly due to genuine reasoning and correct problem-solving, or does it mostly arise from randomness, where the reasoning is incorrect but the final answer happens to match?\n\n3. Given the reliance on GPT-5 for verification, how robust are the correctness checks? Did the model ever mis-verify solutions?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The limits of RL-fine-tuned reasoning models is an important topic, especially given saturation on common benchmarks.\n\n2. The paper is reasonably well written and clearly explains its data-filtering pipeline and evaluation setup. The figures and tables (e.g., dataset construction flow, pass@k comparisons) are well organized.", "weaknesses": "1. The dataset is almost entirely derived from existing public datasets (DAPO-Math-17K, DeepScaleR), using common filtering, for example, deduplication, sampling, verification, etc. The resulting benchmark is a subset of known problems already curated and verified in prior works. The paper does not introduce new problem formulations, annotation schemes, or evaluation methodologies that go beyond what prior datasets or analyses already offer, especially given the availability of broader, harder, and more rigorously verified benchmarks such as MathArena [3], UQ [4] and more.\n\n2. The motivation and methodology closely mirror those of DAPO [1] and DeepScaleR [2], both of which already include difficult math subsets, verification studies, and RL vs. SFT comparisons. These prior works went further by introducing RL frameworks, conducting ablations, and analyzing exploration effects\n\n3. The evaluation simply confirms what is already known, that current RL methods (e.g., GRPO, R1-style) yield limited improvement over base models, especially on problems where base model can barely solve.\n\n4. The authors claim MATH-B encourages “exploration-driven RL,” and can \"move beyond the current plateau\", but provide no evidences show that training with the dataset help model learn new reasoning strategies. Thus, its value as a benchmark remains speculative.\n\n[1] Yu, Qiying, et al. \"Dapo: An open-source llm reinforcement learning system at scale.\" arXiv preprint arXiv:2503.14476 (2025).\n\n[2] Luo, Michael, et al. DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL. 2025, pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2.\n\n[3] Balunović, Mislav, et al. \"Matharena: Evaluating llms on uncontaminated math competitions.\" arXiv preprint arXiv:2505.23281 (2025).\n\n[4] Nie F, Liu K Z, Wang Z, et al. UQ: Assessing Language Models on Unsolved Questions[J]. arXiv preprint arXiv:2508.17580, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)