MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model
Authors: name: Prasanna Mayilvahanan, name: Ricardo Dominguez-Olmedo, name: Thaddäus Wiedemer, name: Wieland Brendel

With the advent of DeepSeek-R1, a new wave of reinforcement learning (RL)
methods has emerged that seem to unlock stronger mathematical reasoning.
However, a closer look at the open-source ecosystem reveals a critical
limitation: with sufficiently many draws (e.g., pass@1024), many
existing base models already solve nearly all questions on widely used math
benchmarks such as MATH-500 and AIME 2024. This suggests that the RL
fine-tuning methods prevalent in the LLM reasoning literature largely sharpen
existing solution modes rather than discovering entirely new ones. Such
sharpening stands in contrast to the broader promise of RL: to foster
exploration and to acquire new skills. To move beyond this plateau, we
introduce MATH-Beyond (MATH-B), a benchmark deliberately constructed to defeat
common open-source models of up to 8B parameters even under large sampling
budgets. Improving performance on our benchmark via RL requires methods that
learn to reason in ways that go beyond base model capabilities in repeated
sampling. Since the problems are drawn from subsets of DAPO-Math-17K and
DeepScaleR datasets, they remain topically equivalent to standard high-school
math. Validating our premise, RL fine-tuned models such as
Nemotron-Research-Reasoning-Qwen-1.5B and DeepScaleR-1.5B-Preview perform
poorly on MATH-B at pass@1024, showing how existing approaches fall
short on tackling harder instances. We hope MATH-B will catalyze
exploration-driven RL approaches that elicit deeper reasoning capabilities. We
release MATH-B at https://huggingface.co/datasets/brendel-group/MATH-Beyond.