CORE: Concept-Oriented Reinforcement for Bridging the Definition–Application Gap in Mathematical Reasoning
Authors: Zijun Gao
Zhikun Xu
Xiao Ye
Ben Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: large language models
mathematical reasoning
conceptual understanding
fine-tuning
robustness
Primary area: foundation or frontier models, including LLMs

Abstract: Large language models (LLMs) often solve challenging math exercises yet fail to apply the concept right when the problem requires genuine understanding. Popular Reinforcement Learning with Verifiable Rewards (RLVR) pipelines reinforce final answers but provide little fine-grained conceptual signal, so models improve at pattern reuse rather than conceptual applications. We introduce $\textit{CORE}$ (Concept-Oriented REinforcement), an RL training framework that turns explicit concepts into a controllable supervision signal. Starting from a high-quality, low-contamination textbook resource that links verifiable exercises to concise concept descriptions, we run a sanity probe showing LLMs can restate definitions but fail concept-linked quizzes, quantifying the conceptual reasoning gap. $\textit{CORE}$ then (i) synthesizes concept-aligned quizzes, (ii) injects brief concept snippets during rollouts to elicit concept-primed trajectories, and (iii) reinforces conceptual reasoning via trajectory replacement after group failures, a lightweight forward-KL constraint that aligns unguided with concept-primed policies, or standard GRPO directly on concept-aligned quizzes. Across several models, $\textit{CORE}$ delivers consistent gains over vanilla and SFT baselines on both in-domain concept--exercise suites and diverse out-of-domain math benchmarks. $\textit{CORE}$ unifies direct training on concept-aligned quizzes and concept-injected rollouts under outcome regularization. It provides fine-grained conceptual supervision that bridges problem-solving competence and genuine conceptual reasoning, while remaining algorithm- and verifier-agnostic.

Published as a conference paper at ICLR 2026
CORE: CONCEPT-ORIENTED REINFORCEMENT FOR BRIDGING THE DEHNNTION-APPLICTTION GAP IN MATHEMATICAL REASONING
Zijun Gao', Zhikun Xu², Xiao Ye Ben Zhou²
University of Illinois Crhhan-Chhappagu Ariizon State University
ABSTRACT
Large language models (LLMM)) often solve challenging math exercises yet fail to apply the concept right when the problem requires genuine understanding. Popular Reinforcement Learning with Verifiable Rewards (RLVR) pipelines re- inforce final answers but provide little fine-grained conceptual signal, so mod- els improve at pattern reuse rather than conceptual applications. We introduce CORE (Concept-Oriented REinforcement), an RL training framework that turns explicit concepts into controllable supervision signal. Starting from high- quality, low-contamination textbook resource that links verifiable exercises to concise concept descriptions, definitions but fail concept-linked quizzes, quantifying the conceptual reason- ing gap. CORE then (i) synthesizes concept-aligned quizzes, (ii) injects brief concept snippets during rollouts to elicit concept-primed trajectories, and (iii) reinforces conceptual reasoning via trajectory replacement after group failures, lightweight forward-KL constraint that aligns unguided with concept-primed policies, or standard GRPO directly on concept-aligned quizzes. Across several models, CORE delivers consistent gains over vanilla and SFT baselines both in-domain concept-cxereise exercise suites and diverse oft-of-domain math benchmarks. CORE unifies direct training on concept-aligned quizzes and concept-injected roll- outs under outcome regularization. It provides fine-grained conceptual supervi- sion that bridges problem-solving competence and genuine conceptual reasoning, while remaining algorithm- and verifier-agnostic. Related code is available at attps://github. com ARC ASU/CORE
sanity probe showing LLMs can restate
INTRODUCTION
Recent LLMs are becoming good at tackling competition-level questions, yet they fall short of conceptual math reasoning beyond applying competition tricks or executing numerical calculations YYang et al. 2024b) Guo et al. 2025a Huang & Yang 2025 Chen et al. 2025). Here, conceptual reasoning means identifying
right concept and applying it in the solution, as opposed to pro- cedural pattern matching often sufficient for GSM8K (Cobbe al. (021) MATH (Hendrycks) tests (Patel et al. [202 Yu et al.) 2024: Mirzadeh al. 2025| Huang et al. 2025). On many benchmarks, models mimic solution templates, chain together routine algebraic steps, and even memorize recurring patterns-wii while still choosing the wrong concept for problem failing to correctly apply certain concepts. This gap matters: solv- ing a word problem by spotting a familiar cue is not the same as understanding linear independence, continuity, or convexity and deploying those notions correctly (Li al. 2025 Huang al.) 2025).
cct al.
2021b) and exposed by perturbation-base
Two factors contribute to the definition-application gap. (i) Exercise-style corpora often reward exploiting surface regularities (formats, keywords, step patterns) rather than applying the intended concepts (Guo et al. [0225]] Wu et al. (2025.. (ii) RLVR pipelines (Schulman et al. 2017 Shao al. 2024) optimize a terminal correctness reward, which is too coarse teach which concept to invoke or how it should support intermediate steps (Qin et al., 2025). Since concepts require goal-
Correspondence Zijun Gao (rjjung3@llinois.eed) and Zhikun Xu Lankunxu@asu.cda

---

Published as a conference paper at ICLR 2026
driven instantiation to be testable, we pair concise definitions with aligned quizzes; models readily recite definitions yet often fail the quizzes.
In order to mitigate this definition-aaplliattoon gap, we propose CORE (Concept-Oriented REin- forcement), an RL-based framework that turns explicit mathematical concepts into concept-driven training signals in sampling. CORE starts from curating a high-quality data that (i) provide human- verified exercises and (ii) link each exercise to the underlying concept(s), which serve as the in- domain test set and seed data for further generation and training signals. We then expand coverage by generating additional concept-aligned quizzes using LLMs to curate the training set. For the training recipe, CORE has explored three designs: Original RL (CORE-Base): directly training with the generated quizzes by RL algorithms. Concept Enhancement (CORE-CR): injecting concise concept snippets into rollouts to replace part of the original ones when all trajectories are incorrect, and KI. Divergence (CORE-KI.) implementing the KI. divergence term between the concept-guides trajectories and the original ones to implicitly constraint the model towards using concepts. The above design choices investigate in three main components, data, rollouts, and loss function, of RL algorithms to mitigate the definition-applicatinn reasoning. Moreover, this framework wraps around standard policy-gradient reinforcement algo- rithms without architectural changes. At test time, the trained model is evaluated without providing the concept text, measuring whether concept-aware training translates into genuine conceptual com- petence rather than reasoning shortcuts.
gap and improve the conceptual
Empirical results show that CORE delivers consistent and significant improvements across Qwen2-Math-7B (Yang et al., 2024a),
(DeepSeek-Al 2025a), al. 2024b) and (lama-3-8B-Instruee (Team) 2024). Across four model families and a wide range of mathematical reasoning benchmarks, CORE and its variants con- sistently deliver substantial performance gains, demonstrating strong mross-model generalization. On Qwen2-Math-7B. CORE variants achieve large improvements, with gains of up to +9.3% % on TEXTBOOK and +9.6% on THEOREMQA, indicating enhanced conceptual alignment and deeper reasoning. For SeepSeek-RI-DO-1... STEM and +1.2% % on SVAMP. On @wen2..5-Math -.5. CORE yields consistent gains, boosting MINERVA MATH by +3.3% and TABMWP by +1.9% %. Finally, on Llama-3-8B- Instrucctt CORE- CR outperforms the Vanilla baseline, achieving up to +3.3% on TABMWP, while also improving results MMLU-STEM and SVAMP. These results demonstrate that, without any architectural modifications, CORE consistently enhances conceptual understanding and reasoning ability through explicit concept injection and concept-aware optimization.
1wen2.5-Math-1.SBB//
CORE-CR improves performance by +1.3% on MMLU-
RELATED WORKS
Mathematical Reasoning in LLMS Recent systems approach math reasoning through special- ized training or sheer scale. WizardMath (Luo al. 2025) uses reinforcement learning from Evol- Instruct feedback. MAmmoTTH (Yue et al.] 2024) blends chain-of-thought and program-of-thought for hybrid tuning. Qwen2.5-Math (Yang et al. 2024b) continues from general model on large curated math corpus. Llemma (Azerbayev et al. 2024) is pre-trained on Proof-Pile. DeepSeekMath (Shao et al. 2024) adds about T20B math tokens and introduces Group Relative Policy Optimiza- fion (GRPO). InterLLM--Math (Ying et al. 2024) unifies
reward modeling, and formal reasoning. General-purposs models also lean heavily into math: Llama 3.1 (Team] 2024) up-samples math data, DeepSeek-RI and DeepSeek-Vy V3 (DeepSeek-Al Al 2025b) raise math and pro- gramming proportions across trillions of tokens, Claude 3.7 (Anthropic) 2025) emphasizes transpar- ent multi-step reasoning, Gemini (Gemini (225)) targets long-chain deduction, and OpenAl ol and o3 (OpenAl 2024; 2025) scale test-time compute. Despite these advances, many pipelines reward fi- nal answers rely on data scale, leaving concept selection and application under-taught. Our CORE addresses this by injecting explicit concept signals into rollouts and regularizing outcomes, yielding consistent gains on concept-dependent evaluations while remaining agnostic to the underlying RL algorithm.
Mathematical Benchmarks A wide range of math benchmarks now probes reasoning at various levels. At the elementary level, GSM8k is the standard for multi-step math problems. MAWPS et al, 2016) aggregates sources, ASDiv (Mio et al.] 2021) adds type and grade labels, and SVAMP (Patel et al. 2021) stresses robustness through controlled perturbations.
2

---

Published as a conference paper at ICLR 2026
Cross-lingual coverage includes CMATH for Chinese primary school Wei et al. 2023) and CN Middle School 24, while standardized suites such as Gaokao 2023 EN. SAT Math, and MMLU- STEM enable broader STEM-wide comparisons (Zhong et al., 2023 Hendryeks et al.) 2021b). For comperition-level reasoning. MATH curates Olympiad and contest problems with stepwise solu- tions, and OlympiadBench (He et al. 2024) extends to bilingual and multimodal settings that em- phasize proof-style reasoning and reduce contamination risk. Our evaluations of CORE are based on in-domain concept-exereise suites and out-of-domain math benchmarks, showing consistent gains over strong baselines and highlighting improvements specifically on concept-dependent categories.
Conceptual Reasoning Answer accuracy alone doesn't reveal whether the right concepts were selected and used, several works probe whether models actually select and use the right concepts. Specifically in math, conceptual reasoning requires people to reason around math concepts and ax- ioms at the play of math hypothesis, statements and problems (Simon 2011). THEOREMQA (Chen et al., 2023) targets theorem application across STEM, explicitly requiring mapping from a named theorem to its correct use in problem solving. GSM-SYMBOLIC (Mirzadeh et al. 2025) exam- ines symbolic generalization limits of models trained on ESM8K--stle data. COUNTERMATH (Li et al. 2025) proposes counterexample-drive
consept-sensitive evaluations to diagnose superficial cues versus true concept use. Complementary directions include a conceptualization framework that maps abstract questions into verifiable symbolic programs (Zhou et al. 2024), supe-supe analogical learning scheme that transfers high-level solutions across cases (Zhou et al.) (2055). and
Bayesian inference formulation coupling abductive proposals with structured deduction for cal- ibrated decisions (Feng et al.) 2025). Overall, these efforts indicate that a single end-point score insufficient for assessing whether models select and correctly apply concepts; concept-aligned training signals or structured evaluation protocols are required.
3 CORE: CONCEPT-ORIENTED REINFORCEMENT LEARNING
3.1 OVERVIEW
We study math reasoning where success depends mainly conceptual math reasoning rather than replaying surface templates the training data. Our framework CORE proceeds through dataset curation, gap diagnostics, and concept reinforcement. For dataset curation. we have leveraged a classical mathematical textbook with clear associations between concepts and exercises for train- ing and evaluation. For gap diagnosties, we have used the curated data both qualitatively and quantitatively identify the definition-application gaps in the conceptual mathematical reasoning. For concept reinforcement recipe, we have mainly designed three training recipes in reinforcing the models' conceptual mathematical reasoning.
3.2 DATASET CURATION
To acquire rigorous conceptual reasoning signals, we curated a corpus from a canonical textbook, Advanced Algebra (3rd Edition) (Yao & Xie 2013). This source was chosen for two-fold reasons. First, it provides a comprehensive and structured curriculum in linear algebra, progressing logically from foundational concepts like determinants and matrices to advanced topics such as linear spaces and canonical forms. Its ten chapters are methodically structured, each containing: i) core concept definitions (C), ii) illustrative examples, and iii) concept-aligned exercises (E), where exercises in given chapter primarily test the concepts introduced in that same chapter. The textbook's 's long- standing use and human verification ensure a logical progression of topics and coherent conceptual dependencies, making it ideal corpus for developing and evaluating concept-oriented learning paradigm. Second, by manually translating this Chinese textbook into English, we significantly mitigate the risk of training data contamination present in many existing laglush- corpora.
The extraction yielded 236 concept texts, 703 examples, and 140 multiple-choice questions sourced from the exercise sections. More details provided in Appendix A
3

---

Published as a conference paper at ICLR 2026
Step 1: Initial Judgment Surface Patter Matching
Step 2: Theorem Recall
Explicit Memorization
User
User
Question: [(x) be polynomial integer coelficients.
Please Rational Root Theorem.
Which of the following statements is true?
A) [()) rational
reducible over the nationals.
reduced Fraction is roxt of f(x), then q divides the
Rational Theorem:
L.IM
constant of (ar).
For [(x) 2t + ... + with integer coeflicients a, é 2. f(2) lowest temms, then p Ro
C) a prime divides all cnefficients of f (a) excepe the leading
[(r) irreuccble.
D) If (() has a repeated factorr, then it must have a repeated ront.
g Bm.
Solve step by step.
Step 3: Re-application
L.M
Still Superticial Reasoning
"If reduced fraction is a root of f(x), then 4 divides the constant term." False. This misstares the Rational Root Theorem. If r is rational rool in lowest terms, then consiant lerm and p leading enefficient.
Statement B:
User
Use Rational Root to solve question again.
So it's backwards: the numerator divides the constant term, not the denominator. statement
LLM
Option B reverses roles of and q. so it is wrong.
Reject
An example of ChatGPT-4o's superficial understanding of the Rational Root Theorem. Although the model correctly recalls the theorem, it fails to engage in structural verification across reasoning steps.
Figure I:
3.3 GAP DIAGNOSTICS
3.3.1 PROBING THE GAP BETWEEN KNOWLEDGE RECITATION AND APPLICATION
The structured nature of curated corpus, with its explicit mapping between concepts (C) and exercises (E), provides an ideal testbed to diagnose a critical failure mode of LLMs: the disconnect between parametric knowledge and its application in problem-solving We conduct the following sanity check to probe this gap, which provides the foundational motivation for our proposed training framework.
We conduct a qualitative analysis multiple-choice exercises that strong baseline, GPT-4o fails due conceptual errors. After an incorrect answer, we prompt the model describe the core concept underlying the problem (e.g., "Describe linear independence.") In most cases, the model correctly recites the definition and key properties, suggesting that the knowledge is parametrically stored. Figure [ shows representative case: GPT-4o accurately states the Rational Root Theorem but still misapplies when the numerator and denominator are swapped. This reveals a defini- tion-application gap: the model can recall concept yet fails to deploy it flexibly in reasoning.
3.3.2 SYNTHETIC CONCEPT PROBES AND ROBUSTNESS EVALUATION
Measuring Conceptual Understanding via Concept Probes. A fundamental challenge in evalu- ating mathematical reasoning is the difficulty of quantitatively measuring a model's grasp of specific concepts. High-level benchmark scores often obscure fne-grained conceptual failures. The highly structured nature of our curated textbook, however, provides a unique opportunity to address this challenge. To create a direct and quantifiable measure of conceptual understanding, the idea of Concept Probes: targeted quizzes generated directly from the textbook's k's concept defini- tions and theorems, where a model's performance on these probes serves as a proxy for its mastery of the underlying concepts.
introduce
To realize this, we constructed new dataset of conceptual quizzes. This process involved two key stages: generation and validation.
1. Generation: We prompted powerful generator model, Qwen2.5-72B Instruct, create quizzzs 5-8 multiple-choice quizzes for each of the 236 concept texts in our corpus. This resulted
http:://ppenai.ccomnneeee
4

---

Robust Evaluation Published as a conference paper at ICLR 2026
in a candidate pool of 1,200 quizzes, each designed to be closely tied to its source concept and formatted with standard LTEX..
2. Validation: To ensure the quality and validity of these synthetic quizzes, we designed rigorous filtering pipeline using separate, powerful assessor model, GPT-4o. This cross- model validation strategy intentionally designed to reduce harvester bias. For each quiz, GPT-4o evaluated six dimensions (e.g., clarity, correctness, uniqueness) and provided an overall rating and confidence level. We discarded the 90 quizzes that were rated "Fair" "Poor" with high confidence. This stringent process yielded final set of 1,110 high- quality quizzes that serve as our Concept Probes.
Diagnostic Experiment: Robustness to Superficial Perturbations. To validate hypothesis that models rely on superficial heuristics, we designed a diagnostic experiment to test the robustness of their conceptual knowledge. Using our 1,110 curated quizzes, we protocol. For each quiz, we generate three variants by randomly permuting the order of its multiple- choice options. A model is considered have robustly solved a problem only if correctly answers the original question and all three of its permuted variants. This protocol is designed to test whether a model's 's understanding invariant semanticalyy-ireleean changes that preserve the core concept.
We applied both the standard and our Robust Evaluation protocols to a suite of contemporary mod- els, including Owen2-Math-77 OLMo-2-7B (OLMo al. 2025), and Llama-3-8B. The results, illustrated in Table [ reveal a stark and consistent performance gap across all models. For instance, while a model like OL.o--2-7B may achieve high accuracy (e.g., > 70%) under the standard proto- col, its performance plummets to below 50% under Robust Evaluation. This significant degradation provides strong empirical evidence for our hypothesis, demonstrating that the models' success is heavily reliant shallow heuristics rather than a deep, structural understanding of the underlying concepts.
Table 1: Performance comparison under Original vs. Robust Evaluation protocol across different models. The best performance in each column is highlighted in bold.
Standard evaluation
Robust evaluation
Model
pass 1 accuracy
self-consistent
pass @ accuracy
self-consistent
Owen-2-Math-7B
74.33%
87.0% +0
45.92% -28.4
76.0% -11.0
OLMo-2-7B
57.83% +0
70.17% +0
36.25% -21.6
44.42% -2588
LLaMA-3-8B
44.75%
70.92% +0
20.25% -24.5
46.75% -24.2
3.4 CONCEPT REINFORCEMENT RECIPE
To bridge the gap between procedural mimicry and conceptual understanding, we propose CORE, il- lustrated in Figure CORE is RL-based framework designed to inject conceptual knowledge into models through any policy gradient based RL algorithm. The core idea to conditionally intervene during training with concept-guided instruction precisely when the model demonstrates a failure in understanding, guiding the policy update towards a more robust, concept-grounded reasoning pro- cess. Our CORE framework mainly consists of the following three design choices, instantiated with GRPO as the standard RL backbone in this paper. For completeness, we also discuss and evaluate PPO-based variant in Appendix C.I
CORE-Base: Standard RL on Conceptual Quizzes. The foundational approach within our framework, CORE-Base, involves training the policy me directly on our curated set of conceptual quizzes (Q) using the standard GRPO algorithm. In this setting, the model learns from the concep- tual data without any further explicit guidance during the training process. This approach measures the model's ability to implicitly Iearn concepts from the rich question-answe answer pairs generated from concepts.
CORE-CR: Concept-Guided Trajectory Replacement. Building upon the base setting, CORE- CR introduces a conditional intervention triggered by a conceptual failure event (i.e., all N re- sponses in GRPO group incorrect). Upon triggering, form concept-guided prompt
5

---

Published as a conference paper at ICLR 2026
Concept-Guldee Intervention (CORE) Subsystem
Concept Recell:
Cancept Injectian:
CORE-CR [Tayactory Repacemmnt
CORE-KL (Comcspt-AL Motp Alggmmant)
(from librar)
te mode 1y
Reptaoe failed trajectories concept- srimed unes Thonarl
--... bose polkcy 14t
corecot cereeot ma 15a)
concost inaatiannatatto (323300
Fieppramp the Policy Model generats & conceep-- primed replazze
after COPE RL bligned)
A K
xnA ModFFad Battnn
netrivve assonlated conceett
trajectaries (ta) NV
Origial Batch
txpoectoriss Where i...c.a
Pullyy Uistrbukkon
Dau (#a 4) ((flp)))
Incemeet Corret
Yea (Titggered)
Forwars R.. aiggnment swaar
Trajectry
concest-quitedd detribattin
Indectoites incorme?s
(Nol lilgceoo)
added KL loss
Triggred Flow Rath
S Canocale Group
Pore/o Updateed Trajectory Group
Guety
Standard PL Lpdata (CORE (ese)
Polley Mindll (nc)
Figure 2: Overview of the Concept-Guider Reinforcement (CORE) framework. For given query, the policy model generates multiple candidate solutions. If any solution is correct, CORE-Base proceeds. When all solutions fail, CORE activates concept-guided correction: the Concept Recall module retrieves relevant domain knowledge, and Concept Injection re-prompts the model with this guidance to form corrected trajectories. CORE-CR replaces failed paths with these concept-grounder ones to recover the learning signal, while CORE-KL distills the concept-enhanced trajectories into the base policy via a forward KL loss.
Pe - q where q is the original problem from our quiz dataset, Cg is its associated ground-truth concept text, and @ denotes concatenation. We then generate K new trajectories ffe. Tc. from the concept-guided policy, where 1 < K N. We then randomly select and replace tra- jectories from the original failed group with these new concept-guided ones. To incentivize learning with concepts, we assign the new trajectories an augmented reward:
R' R'(Te)) R(Te.) + Tbonus
where Tboms > is hyperparameter. The GRPO update is then performed on this partially replaced, concept-guided batch. Notably, there is a recent work called BREAD (Zhang et al. 2025), which shares very similar methodology to CORE-CR while derives from rethinking the advantages of SFT and RL instead of improving conceptual reasoning. However, as empirically demonstrated in Section 5.4] CORE achieves significant gains without distilling reasoning signals from larger teacher model. This autonomy distinguishes CORE from mainstream "expert-anchored" approaches like BREAD, which typically require superior external models to provide trajectory guidance.
CORE-KL. Concept-Guided Kl-RReularization. The CORE-KI. method introduces a fine- grained regularization signal guide the policy's internal reasoning process. This approach is also triggered by conceptual failure event. Instead of directly replacing trajectories, this method encour- ages the model's standard see--by-step predictive process at each timestep f, denoted no(- q. Vet), to align with the more robust process it exhibits when primed with concept, To(- Pe. Veee).
We formulate this as a forward KL-divergence objective. This choice deliberate: it encourages the base policy to cover the full distribution of reasoning paths considered by the concept-guided "teacher" policy, rather than collapsing to a single mode, fostering a more comprehensive distillation of the entire reasoning process. Upon conceptual failure trigger, we first sample a high-qualig, reference trajectory, Y = (yi..... ) from the current, online concept- guided policy, i.c., Y To(- p.). Our objective is then minimize the KL-divergence between the next-token predictive distributions of the guided and un un-guided policies at each timestep t, conditioned on the prefix of the reference trajectory y This is formulated loss term added to the base RL objective:
Liotal LGRPO + Ey (
Σ DEL Te(-
Pe, v22e)
TT(: q. (a.22e)
(I)
where Te(- context, )22) is the current policy's probability distribution over the next token. This forces the model's internal reasoning process on the original problem g to faithfully mimic the process it would follow if it were explicitly given the concept Cq-

---

Published as a conference paper at ICLR 2026
CORE-Base primarily functions consolidation mechanism, reinforcing the application of con- cepts already encountered during pre-training, whereas CORE-CR and CORE-KL serve as comple- mentary corrective strategies for concepts that are not yet reliably mastered, operating in parallel through explicit and implicit forms of concept intervention. Together, these variants constitute paral- lel and complementary ways of incorporating conceptual signals into reinforcement learning within a unified framework.
EXPERIMENTS
4.1 BASELINE MODELS
We select Qwen2-Math-7B as our primary evaluation model. This choice is motivated by its mod- erate mathematical proficiency: it demonstrates non-trivial reasoning skills while still leaving suf- ficient headroom on our quiz tasks, thus offering a reliable foundation for assessing the impact of CORE. For this model, we report results for all three CORE variants (CORE-Base, CORE-CR, and CORE-KL). Beyond this, we further evaluate CORE-CR on DeepSeek-Ro RI-Distill--wen---.. Qwen2.5-Math-11 and Llama-3-8B-nng --Istrutt, highlighting the algorithm'' 's robustness and cross- model generalization across both math-specialized and instruction-uun settings.
4.2 TRAINING SETTINGS
Our CORE variants introduce method-specific hyperparameters: reward bonus Thomus for CORE-CR and dynamic KI. coefficient λKI. for CORE-KI., All training configurations and extensive hyperpa- rameter details are deferred to Appendix B.1
4.3 EVALUATION SETTINGS
In-domain Test. To measure in-domain performance, the 140 multiple-choice exercises curated from the textbook. These exercises serve as a high-quality and reliable measure of concept application due their expert authorship and direct alignment with the textbook's definitions. We denote this test set as Textbook.
Out-of-domain Benchmarks. To assess whether the conceptual understanding fostered by CORE generalizes beyond our curated training data, we evaluate our trained models on a diverse suite of out-of-domain benchmarks. These benchmarks of mathematical reasoning, from multi-step arithmetic and competition math to robustness against perturbations. This evaluation is critical demonstrate that CORE. does simply overfit to the textbook's s style, but rather instills more fundamental and transferable reasoning capability.
specifically chosen to probe for different facets
We evaluate models trained with three instantiations of our CORE CR, and CORE-KL-on the following out-of-distribution benchmarks: GSMSK (Cobbe et al. ASDiv (Miao al., (202T) MAWPS (Koncel-Kedziorsk al., (0166), TabMWP (Lu al.
wwwp:/owoo--RRRBBBa.e CORE-
2021,
2023
MATH (Hendrycks et al. (0211), MMLU-STEM (Hendrycks et al. 2021a). Gaokao 2023
(EN)
[Zhong et al. 2023), Gaokao-Mathh
(Zhong al.) (2023), CMATH (Wei et al. 2023)
Minerva Math (Lewkowycz et al. (022). SVAMP (Patel et al. (021).. CounterMath (Li et al.)
2025). TheoremQA (Chen et al.) 2023), and OlynpiadBench (He et 2024). A detailed descrip-
tion of the datasets is provided in Appendix B.2
Metrics. All results are reported with self-consistency (SC@21, T = 0.7); Appendix B.3
As shown Table 2 models trained with the CORE framework exhibit consistent and significant the majority of out-of-domain benchmarks when compared to the vanilla baseline. This demonstrates that CORE successfully enhances the models' underlying reasoning abilities in way that generalizes unseen problem distributions and formats.
performance improvements

---

Published as a conference paper at ICLR 2026
Table 2: Main table accuracy (%) under SC@21 (T=007). Columns use two-
letter abbreviations: TB-Textbook, GS=GSM8K, AD=ASDiv, MW=MAWPS, TM=TabMWP,
MH=MATH, MS-MMLU-STEM, GK=Gaokao 2023 (EN), CM=CounterMath (reported as F1),
TQ=TheoremQA, O1=OOlympiadBench
Model
Method
TB
GS
AD
MW
TM
MH
MS
GK
CM
TQ
OL
Vanilla
46.4
89.8
95.1
96.8
90.2
69.1
72.9
55.3
13.2
34.6
28.7
SFT
45.0
86.6
94.1
96.6
85.6
59.4
72.4
46.5
16.7
44.2
19.7
Qwen2-Math-7B
CORE-Base
50.7
90.8
95.4
97.2
92.6
71.1
72.9
59.5
13.5
40.4
33.9
CORE-CR
52.1
91.1
95.7
97.3
93.6
71.4
72.6
58.4
15.5
42.3
34.5
CORE-KL
55.7
90.7
95.5
97.5
90.6
70.5
73.1
59.5
15.8
44.2
32.9
Table 3: Results on the diagnostic subset W (W] = 19). problem is Concept-Selection iff both CORE-CR and CORE-KL explicitly invoke the target concept and show heuristic cues; Heuristic- selection iff both rely on heuristics with no substantive concept use; otherwise Mixed.
Category
Problems
Probability %)
Concept-selection
10
52.6
Mixed
47.4
Hcuristic-selection
0
0.0
Total
19
100.0
5 ANALYSIS
5.1 DOES CORE ENHANCE CONCEPT SELECTION AND APPLICATION?
We first verify what training with CORE actually changed: are the observed accuracy gains at- tributable improved concept selection and application, rather than superficial heuristies, and achieved without any test-time concept prompting? To probe this, we evaluate four model variants (Vanilla, CORE- Base, CORE-CR, and CORE-KL) 140 textbook exercises, and define diagnostic subset W' comprising problems that are solved by both CORE-CR and CORE-KL but not solved by either Vanilla or CORE-Base.
W = ( (Vanila fails or CORE-Base fails on (CORE-CR and CORE-KLL succeed i) }-
yielding [W| = 19 (12 Vanilla-only failures, CORE- Base-only failures, shared failures). For each i W. we read the generations from CORE-CR and CORE-KL and score along two dimensions: (i) concept hits-the output explicitly mentions the task's target concept and uses it correctly in the reasoning; and (ii) heuristic cues quesssnessing, option elimination without justification, surface pattern matching, or plug-in substitution without conceptual warrant.
For labeling, we keep the rules simple. A problem is called Concept-Selectinn if both CORE outputs (CORE-CR and CORE-KL) contain a concept hit and neither shows heuristic cues; it is classified as Heuristic-Selection if both outputs rely on heuristics and contain no concept hit; otherwise, categorized as Concept+Heuristic (mixed). We require "two hits" (one per CORE variant) so that if either variant omits the concept, the instance is not counted as Concept-Selection, As Ta- ble 3 shows, 10/19 (52.6%) cases are Concept-Selection. 9/19 (47.4%) are Mixed, and 0/19 are Heuristic-selectnn
Due to our strict constraints, it rules out superficial shortcutting as the primary driver the gains. On one representative question (see Table 9). even after appending a targeted concept prompt at test time, CORE-Base remained incorrect, whereas CORE-KL solved it without any prompt. While this a single illustrative case, it reinforces that the observed gains come from induced mechanism change rather than prompt engineering. Taken together, the evidence indicates a mechanism shift: CORE improves accuracy mainly by strengthening concept selection and application.
8

---

Published as a conference paper at ICLR 2026
Performance "Common" Metrics
Performence "Indvidual"
100.0%
100.0%
95.0%
66.0%
Cla Poseeee
50.0%
Accurwty (5)
Accusty
60.0%
79.0%
75.0%
70.0%
75.0%
65.0%
85.0%
0.
K Vabin
K Value
Figure Performance comparison on Common vs Individual metrics.
5.2 DOES CORE IMPROVE ROBUSTNESS TO IRRELEVANT CONCEPT PERTURBATIONS?
We next ask whether training with CORE yields improved robustness to irrelevant concept cues. Using 140 high-quality, thematically related textbook exercises, we prepend K 2,3 concepts that are not directly related to the target concept to each question and measure whether the model can still retain the correct answer under such perturbations. To ensure that distractors are not directly related, select concepts drawn from different textbook chapters. For each question and each K, we sample one fixed distractor set once (single random seed) and use the same set for all models to enable paired comparisons.
To quantify retention under perturbation, we report RUD (Retention Under Distractors): accuracy on perturbed items restricted questions model already solved without perturbation. Formally, letting Sm be the questions solved by model m the unperturbed setting, x; (K) be the question with K distractors prepended and y/ be the correct label,
(UDx(m)
Σ iESm
{{
(m((((4))) m a
Vi
S
We evaluate four models-Vaninn CORE-Base, CORE-CR, and CORE-KL-under two splits: Com- mon (items solved by all models; = 48) and Individual (per-model solved sets: Vanilla 65 / CORE-Base 71 CORE-CR 73 CORE-KL 78). The resulting RUD k curves for each split can be seen in Figure 3
As increases, models trained with CORE show consistently smaller accuracy drops than Vanilla and CORE-Base both splits, with the CORE-CR variant particularly robust. This pattern indicates that CORE not only improves the accuracy the headline, but also improves the robustness, making predictions more stable against perturbations of irrelevant concepts.
5.3 DOES CORE APPLY TO BASE AND INSTRUCTION-TUNED MODELS?
We ask whether these gains persist across both base and instruction-tuned models. We therefore ap- ply CORE-CR to three representative models: DeepSeek-ER -aoeek--R--B --Dssiill-Qee--L-e Qwen2.5-Math- 1.5B, and Jlama-3-SB-Instruct We evaluate them under the same SC@21 (T'=0.7) protocol across
diverse out-of-domain suites (Table abbreviations follow the table).
CORE-CR yields consistent average improvements across all three models: DeepSeek-RI-DQ-11 improves from 72.7 + 73.1 (+0.4), Qwen2.5-Math-1155
1.5h from 72.1 - 72.4 (+0.3), and Llama- 3-8B-Inst from 58.1 - 58.9 (+0.8). Taken together, these results indicate that CORE is model- agnostic, providing consistent gains across both base and instraction-tuned models.
5.4 CORE DRIVEN BY KNOWLEDGE DISTILLATION OR INTRINSIC CONCEPTUAL REINFORCEMENT?
critical concern in RL training with synthetic data is whether the observed gains stem from simple knowledge distillation from superior teacher model. To disentangle CORE's mechanism

---

Published as a conference paper at ICLR 2026
Qwen2.5-Math-1.55 and Llama-3-8B-Insrrutt Out- of-domain benchmark accuracy (%) under SC@21 (7=0.7). Columns use two-letter abbrevia-
Table 4: PeepSeck-RI--Dstiill--wawe
tions:
CA=CMath, GQ GaokaoMathQA, GK=Gaokao 2023 (EN), MH-MATH, MW=MAWPS,
MM-Minerva Math, MS=MMLU-STEMM, SV=SVAMP, TM=TabMWP
Model
Method
CA
GQ
GK
MH
MW
MM
MS
SV
TM
Vanilla
90.8
75.2
58.2
68.6
96.9
23.9
58.6
92.8
89
DeepSeek-RR1-D-11..B
CORE-CR
91.5
75.5
59.2
69
97.1
24.3
59.9
87.6
Vanilla
60.7
59.5
75.9
97.1
26.1
61.2
93
84
Qwen2.5-Math-Ia.
CORE-CR
91
57.8
60
77.2
97.6
29.4
59.4
93.3
85.9
Vanilla
78.8
25.9
35.8
41.6
93.8
16.9
63.2
90
77.1
-lama-3-B--mm
CORE-CR
79.7
26.2
36.6
39.9
95.4
15.8
64.6
91.6
80.4
Table 5: Results of the Self-Supervised experiment using Qween--aatt-7--Instuutt as the gener- ator and Qwen2-Math-7B as the learner. Abbreviations: AD=ASDIV, GK=GAOKAO 2023 (EN), GS-GSM8K, MH=MATH, MS=MMLU-STENN MW=MAWPS, OL-OLYMPIADBENCHH
TM=TABMWP.
Benchmark
GS
AD
MW
TM
MH
MS
GK
OL
Vanilla
89.8
95.1
96.8
90.2
69.1
72.9
55.3
28.7
CORE-CR
91.4
95.5
97.6
92.5
70.4
73.1
57.7
32.9
from such effects, we conducted a "Self-Supervised" experiment by restricting the entire pipeline to the Qwen2-Math-7B family, thereby eliminating any external "expert" guidance from ultra- large LLMs. Specifically, we utilized Qwen2-Math--BB-nstte on the 236 textbook concepts. To ensure quality without external supervision, we employed self-veriffcatiio protocol where only the 670 quizzes that the model could solve correctly via self- consistency SC@21) were retained.
generate 885 quizzes based
As shown in Table 5 CCRE-CR delivers consistent and substantial improvements across bench- marks even with self-generated and potentially noisy concept probes. This evidence suggests that CORE's efficacy does not depend on perfectly clean quiz data or a superior expert model. Instead, the framework is naturally robust, capable of extracting useful learning signals through its intrin- sic intervention logic, effectively bridging the reasoning gap through a closed-loop, self-improving process. To further disentangle whether the observed performance gains arise from intrinsic concep- tual reinforcement rather than reinforcement learning artifacts or passive process supervision, we provide extensive ablation studies in Appendix C.2
CONCLUSION
In this work, we introduced CORE (Concept-Oriented REinforcement), reinforcement learning framework designed to bridge the gap between conceptual definition and practical application in mathematical reasoning. By leveraging a curated concept-exereise exereise corpus, we diagnosed the limi- tations of current LLMs and injected explicit concept signals into the training process. Specifically, through concept-aligned quizzes, concept-guiee guided trajectory replacement, and KL-based divergence regularization, CORE delivers granular supervision that extends beyond simple outcome correct- ness. Extensive experiments on both base and instruction-tumm models demonstrate that CORE consistently yiclds substantial performance gains across in-domain and out-of-domain benchmarks, particularly in conceptual precision and robustness. Crucially, these improvements are achieved without architectural modifications, ensuring seamless compatibility with standard policy-gra aoriiy-gradient methods. Our findings underscore that grounding reinforcement learning in explicit concepts can transcend surface-level heuristics, moving LLMs toward genuine conceptual competence. We hope this work motivates further exploration of concept-centered training signals in mathematics and other domains where principled reasoning paramount.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our dataset is curated from high-quality educational resources originally published in Chinese, We contacted the primary author, who indicated they could not grant permission at this time due to unclear regulations around LLM training and evaluation in Copyright Law of China. After consult- ing legal guidance, we understand that limited use of such materials for non-commercial academic research may be permissible. Accordingly, our use is strictly for research and education; we do not redistribute substantial verbatim text. The rescarch artifacts (codes, prompts, scripts, structured concept-exereise mappings, and model-generated quizzes/snippets) are derived and only small il- lustrative samples, that does not contain substantial portions of the original expression, would be presented. We cite sources and will promptly honor takedown or correction requests. Any released artifacts are for research use only and may not be used commercially; parties seeking commercial use should contact the rights holders. This statement is not legal advice, and we will adjust our practices as regulations evolve.
REPRODUCIBILITY STATEMENT
Our proposed framework is specified in 3 Our data curation, motivation verification, and training recipes are illustraed under it with subsections named as Dataset Curation, Gap Diagnostics, and Concept Reinforcement Recipe. The training and evaluation settings appear in $4 and Appendix B We will provide a full code repo with codes, configs, and scripts to run training and evaluation end-to-end in the camera-ready version.
REFERENCES
Anthropic. Claude 3.7 sonnet and claude code. Anthropic News, 2025. URL httpt::////// wwww. anthropic. com hwtmm//ww//ccllcc cawee///laauee33 -on-onneet Accessed: 2025-09-22.
sonnet.
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Al- bert Q. Jiang, Jia Deng. Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics, 2024. htt.hhtss:///arx.
wator//ab///33/ 10631
Luoxin Chen, Jinming Gu, Liankai Huang, Wenhao Huang. Zhicheng Jiang, Allan Jie. Xiaoran Jin, Xing Jin, Chenggang Li, Kaijing Ma, Cheng Ren, Jiawei Shen, Wenlei Shi, Tong Sun, He Sun, Jiahui Wang, Siran Wang, Zhihong Wang. Chenrui Wei, Shufa Wei, Yonghui Wu, Yuchen Wu, Yihang Xia, Huajian Xin, Fan Yang, Huaiyuan Ying. Hongyi Yuan, Zheng Yuan, Tianyang Zhan, Chi Zhang, Yue Zhang. Ge Zhang, Tianyun Zhao, Jianqiu Zhao, Yichi Zhou, and Thomas Han- wen Zhu. Seed-prover: Deep and broad reasoning for automated theorem proving, 2025. URL abss https://arxiie wwwpxivv.org/aaae abs / 2507 23726
Wenhu Chen, Ming Yi,, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. TheoremQA: A -drieen-driven question answering dataset. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Meth- ods in Natural Language Processing, pp. 7909-7901, Singapore, December 2023. Associa- tion for Computational Linguistics. doi: 10. 186538/3/ 1/tp:////22////// 1/2023..mmll-main... emnlp-main.489) URL https: //aclanthology.orgg
emnlp main. 489/
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers solve math word problems. 2021. URL https://arxiv.
org/ htg//bs//20 22100 14168
DeepSeek-AL. Deepseck-rl: Incentivizing reasoning capability in lims via reinforcement learning, 2025a.
wttpv//oogg//aaaa//eooeee 12948
DeepSeek-AL. Deepseek-v3 technical report, 2025b. URL http://ps///arvi..o///a////////
19437
Yu Feng, Ben Zhou, Weidong Lin, and Dan Roth. Bird: A trustworthy bayesianninfeeeenne inference framework for large language models, 2025. /t//:tttess//rrrvv..or///aa////p 12494

---

Published as a conference paper at ICLR 2026
Team Gemini. Gemini 2.5: Pushing the frontier with advanced reasoning. multimodality, long context, and next generation agentic capabilities, 2025. URL https://arxxt wrw. abs/
2507.06288
Dadi Guo, Jiayu Liu, Zhiyuan Fan, Zhitao He, Haoran Li, Yumeng Wang, and Yi R. Fung. Math- ematical proof as litmus test: Revealing failure modes of advanced large reasoning models, 2025a. URL
Jiaxing Guo, Wenjie Yang, Shengzhong Zhang. Tongshan Xu, Lun Du, Da Zheng, and Zengfeng Huang. Right is not enough: The pitfalls of outcome supervision in training Ilms for math rea- soning, 2025b. Jnttp::///aazi wttp://wwww....aaa//....... 06877
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi Zhiyuan Liu, and Maosong Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scien- tific problems. In Lun-Wet Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pp. 3828-3850, Bangkok, Thailand, August 2024, Association for Computational Linguis- tics. doi: 10. (038655//1/2224..a 222/222..2l-2oo.22222
URL https://eellattaoooo..rr////////
acl acl-long.2117
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. Measuring massive multitask language understanding, 2021a. URL. https:
/arxiv.org/abb..
03300
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardr. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021b. URL BtRpstps::20 httttp:://ppeneevee...... wtwp://wwwweeeoooooneeeeo hartt//orrmmmicc BBytt
Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge. Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Yuan, Runzhe Wang, Yue Wu, Ming Yin, Shange Tang, Yangsibo Huang, Chi Jin, Xinyun Chen, Chiyuan Zhang, and Mengdi Wang. MATH-perturb: Benchmarking LLMs' math reason- ing abilities against hard perturbations. Forty-second International Conference on Machine Learning, 2025. /ttp:
net/ fastt/forum200 -itimmiddddiyioiii 70UggxX
Yichen Huang and Lin E. Yang. Gemini 2.5 pro capable of winning gold at imo 2025, 2025, URL https://arxiv. www.ooogg//oooooooo abs///00//20//// 15855v3
Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. MAWPS: A math word problem repository. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 152--117, San Diego, Califor- nia, June 2016. Association for Computational Linguistics. doi: 10. 10.18653/11/116- 1 136. URL ottps::/aclLatthology. wtwphhhoooooyoor//NNe. 1136/
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Sole Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with lan- guage models, 2022. URL https::////.. /ttpiv..vvvvv////////e./aaaaa org/ 2206. 14858
Yinghui Li, Jiayi Kuang, Haojing Huang, Zhikun Xu, Xinnian Liang, Yi Yu, Wenlian Lu, Yangn- ing Li, Xiaoyu Tan, Chao Qu, Ying Shen, Hai-Tao Zheng, and Philip S. Yu. One exam- ple shown, many concepts known! counterexample-drive conceptual reasoning in mathemat- ical LLMs. In Forty-second International Conference on Machine Learning, 2025. URL
https: http:://peereveiieee net/ Eorum 2orummidd=A31122210
Pan Lu, Liang Qiu, Kai-Wai Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for stmi-sttrctured mathematical reasoning. In The Eleventh International Conference on Learning Representations, 2023. htt.htttss//opereeee. ew net. www.:ete.forrmmtteoeekyyro.
12

---

Published as a conference paper at ICLR 2026
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, and Dongmei Zhang. Wizardmath: Empower- ing mathematical reasoning for large language models via reinforced evol-instruct, 2025. URL btttps://arxiv www.ooggg/bb/ooooooo//o.//eeaee/ abs abs/2308 2308 09583
Shen- Yun Miao, Chao-Chun Liang, and Keh- Yih Su. A diverse corpus for evaluating and developing http://xxx....//aaa//...........
english math word problem solvers, 2021. URL
Iman Mirzadeh. Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. Gsm-symbolic: Understanding the limitations of mathematical reasoning large language models, 2025.
wwwp////gggbb/boeoooooooo 2410 05229
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bha- gia, Yuling Gu. Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson. Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Allyson Eurnnger, Michal Guerquin, David Heineman, Hamish Ivison, Pang Wei Koh, Ji- acheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam. Jake Poznanski. Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. 2 olmo 2 furious, 2025. URL https:
f/arxiv.orq// abs / httpaa//5500.00000 00656
OpenAl. Openai ol system card, 2024. URL
OpenAl. Openai o3 and 04-mini system card.
OpenAI, 2025. URL
https:
wwc.ppppenaa..cmmppaa////2eeeeee ctm
02dc 4789- -00-8000-2777532222111111
10-an--of-minl--yysttmm---
pdf
Accessed: 2025-09-22.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve sim- ple math word problems? In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techmologies, pp. 2080-2094, Online, June 2021. Association for Computational Linguistics. doi: 10. 10.188/311122// .nnacl-maiit 168. URL httts::/aaclante oogy wwt.yyorrg/22220 nwa.aalll-e.....m ma i n. 1...1666 168/
Tian Qin, Core Francisco Park, Mujin Kwun, Aaron Walsman, Eran Malach, Nikhil Anand, Hidenori Tanaka, and David Alvarez-Melis. Decomposing elements of problem solving: "mat"math does teach?, 2025. URL https: http::///aaxv...o// wwwpxxxwwwwggabb//x/p
22756
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL http://ss//aaii..oog////////////
7777006347
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024.
03300
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rihf framework. 2024. URI.
/tttps://axxi.. /tt//aiivv.// http://aaxvvv.v..aaa/... org/ httgg//bb//443/33 2409. 19256
Martin A Simon. Studying mathematics conceptual learning: Student learning through their math- ematical activity. North American Chapter of the International Group for the Psychology of Mathematics Education, 2011. URL https: (00........gg///...55555556
LLaMA Team. The llama herd of models, 2024. URL httpp::///axii..rr///bbb/
21783
Tianwen Wei, Jian Luan, Wei Liu, Shuang Dong, and Bin Wang. Cmath: Can your language model pass chinese elementary school math test?, 2023. URL http::///xxxvv.r////o haxxiv.org//bs//3006 abs/g//6.
T6636
13

---

Published as a conference paper at ICLR 2026
Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Huijie Lv, Ming Zhang, Yanwei Fu, Qin Liu, Songyang Zhang, and Qi Zhang. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination, 2025.
htt.httts:///axiiii
abs/ 2atas/2507 2507. 10532
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li. Dayiheng Liu, Fei Huang. Guanting Dong, Haoran Wei. Huan Lin, Jialong Tang, Jialin Wang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jin- gren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wen- bin Ge. Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024a. URL httpps:/aare
otg///////ooooooooooo abs/ 2407.1066
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5- - technical report: Toward mathematical ex- pert model via self-immpnn improvement, 2024b. URL http::ttt///. //arxiv org/ abs/ 2409 12122
Musheng Yao and Qihong Xie. Advanced Algebra (Chinese 3rd Edition, Exercise), Fudan University Press, 2015. ISBN 97873091 URL httpst wwtp::/wwwwg..cooooooo
US
Huaiyuan Ying. Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, Yudong Wang, Zijian Wu, Shuaibin Li, Fengzhe Zhou, Hongwei Liu, Songyang Zhang, Wenwei Zhang, Hang Yan, Xipeng Qiu, Jiayu Wang, Kai Chen, and Dahua Lin. mntemllmmmath: math: Open math large language models toward verifiable reasoning, 2024.
2240. 06332
Xiaodong Yu, Ben Zhou, Hao Cheng, and Dan Roth. Reasonagain: Using extractable symbolic pro- grams to evaluate mathematical reasoning, 2024. URL htt.:/ts//aarvvv.or///bbb/////// 19C56
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MAmmoTH: Building math generalist models through hybrid instruction tuning. In The Twelfth International Conference on Learning Representationss 2024. https:::::/ httttp:://openevviee
2et//Eorumm1ddyyLLGG677701
Xuechen Zhang, Zijian Huang, Yingcong Li, Chenshun Ni, Jiasi Chen, and Samet Oymak. Bread: Branched rollouts from expert anchors bridge sft & rl for reasoning, 2025. URL https://
arxiv. or..
17211
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models, 2023. URL httttps//aaxiiv.orr///a/ba
Ben Zhou, Hongming Zhang, Sihao Chen, Dian Yu, Hongwei Wang, Baolin Peng. Dan Roth, and Dong Yu. Conceptual and unbiased reasoning language models, 2024. URL https://
wrxivvorg/abs/ 2404. 00205)
Ben Zhou, Sarthak Jain, Yi Zhang, Qiang Ning. Shuai Wang. Yassine Benajiba, and Dan Roth. Self- supervised analogical learning using language models, 2025. URL https::///// http:///axxxvvooo.ee abs/ 2502..50.... 00996
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A DETAILS FOR TEXTBOOK DATA
A.1 DATA CURATION DETAILS
Our data curation followed multi-stage pipeline ensure high fidelity. We first employed an OCR too to digitize the textbook. The concept and exercise sections then underwent manual verifi- cation stage, with any recognition errors corrected using GPT-4o. Subsequently, the entire Chinese corpus translated into English via GPT-4o, followed by another round of human verification the key sections to ensure accuracy.
B EXPERIMENT SETUP
B. MORE TRAINING DETAILS
In the main experiments, we train models for 3 epochs on four H200 GPUs with GRPO by the verl (Sheng et al. 2024 framework. In experiments, CORE-KL, we use ÀKL 0.03 when the reference concept-guided trajectory is correct, and λKL = 0.005 otherwise. The policy is optimized using Adam with an actor learning rate of 10 and standard KI -ivdivereence penalty with fixed coefficient of 0.001 is applied against the reference policy. During training, we set the sampling temperature to 0.7, with a batch size of 128 and mini-batch size of 32. For each prompt, four responses generated conduct GRPO updates. Rewards follow a binary scheme (1 for correct, 0 for incorrect). The maximum prompt length is capped at 1024, while the maximum response length varies across models, being set to 1024 for Qwen2-Math-P5 7B, 2048 for Qwen2.5-Mathe 1.5B and Llama-3-8P --s--uuttt and 6000 for DeepSeek- RI RI-Distill-Qwen- 1.5B accommodate their extended reasoning contexts.
Tbesss 0.4 for CORE-CR. For the
B.2 DETAILS OF EVAULATION DATASETS
All evaluation datasets sourced from the Qwen2.5-Math evaluation repository Their details summarized below.
GRADE SCHOOL & MIDDLE SCHOOL LEVEL.
GSM8k:
dataset of approximately 8,500 high-quality, linguistically diverse elementary school math word problems. These problems require 2 8 steps of reasoning to solve and primarily involve basic arithmetic operations (+, X, +). The purpose of this dataset is evaluate model's ability perform multi-step mathematical reasoning.
ASDiv: An English math word problem dataset that diverse in both linguistic patterns and problem types. It aims to comprehensively evaluate the true capabilities of math problem solvers, preventing models from achieving high scores merely by "memorizing" solutions to similar problems. Each problem is annotated with its problem type and grade level.
MAWPS: A collection of several thousand English math word problems sourced from var- ious online educational websites. Its goal is to provide an extensible repository math problems for researchers to and expand upon, covering variety of basic arithmetic and algebraic problems.
TabMWP: A large-scale dataset containing over 38,000 math word problems, distinguished by the inclusion of table context for each problem. To solve these, model must be able to retrieve, integrate, and perform multi-step mathematical reasoning on information from both textual and tabular sources.
CMath: A Chinese-language dataset designed to evaluate language models on elementary school mathematics. It contains carefully curated word problems covering fundamental
http::///githbbb.cmmm//eeem
15

---

Published as a conference paper at ICLR 2026
arithmetic operations (addition, subtraction, multiplication, and division) and simple logic- based reasoning. The dataset challenges models to perform precise quantitative reasoning in Chinese, testing both mathematical competence and cross-lingual generalization.
SVAMP: A benchmark consisting of simple arithmetic word problems created by perturb- ing examples from the well-known MAWPS dataset. is designed assess the robustness and true reasoning ability of language models by introducing variations that discourage rote memorization. Models must comprehend problem semantics and adapt to diverse formula- tions of similar mathematical tasks.
HIGH SCHOOL LEVEL
MATH: A dataset created by Dan Hendrycks et al., containing 12,500 problems from American high school math competitions (such as AMC 10, AMC 12, AIME). The prob- lems cover multiple subjects including algebra, geometry, number theory, and counting & probability. Each problem includes a detailed solution written by human expert in ETEX format. Its difficulty is significantly higher than elementary school problems, making it key benchmark for advanced mathematical reasoning.
MMLU-STEM: MMLU includes 57 different subjects, and MMLU-STEM refers to the subset of subjects related to STEM (Science, Technology. Engineering, and Mathematics), such as college-level math, physics, chemistry, and computer science.
Gaokao2023En: This dataset is derived from China's "Gaokao" (National College Entrance Examination) mathematics papers. It typically involves translating Chinese math problems into English to test large model's ability to solve difficult math problems from different cultural and educational backgrounds.
GaokaoMathQA:
high-school level benchmark constructed from authentic Chinese Na- tional College Entrance Examination (Gaokao) mathematics questions. The dataset in- cludes both multiple-choice and open-ended problems covering a broad range of high- school topics, such as functions, geometry, probability, and calculus. It aims to evalu- ate models' abilities to perform symbolic reasoning and multi-step quantitative problem- solving under exam-style constraints.
COLLEGE LEVEL & BEYOND
university-Ievel mathematical benchmark designed to evaluate a model's conceptual reasoning by requiring it to prove or disprove statements by providing coun- terexamples. It focuses on advanced topics Algebra, Topology, Real Analysis, and Func- tional Analysis.
CounterMath:
TheoremQA: A theorem-driven question answering dataset created to evaluate an AI model's ability apply scientific theorems to solve challenging problems. It contains 800 questions covering over 350 theorems from Mathematics, Physics, EE&CS, and Finance.
Minerva Math: A benchmark derived from the work on the Minerva model (Lewkowyez eet al., (2022), which was designed train large language models on scientific and math- ematical content to enable step-by-step quantitative reasoning. The benchmark covers undergraduaae- math and science questions in natural language and LaTeX, requir- ing models to correctly parse, compute, and symbolically manipulate expressions without external tools.
COMPETITION LEVEL
Olympiad Bench: A benchmark of extremely challenging. Olympiad-level scientific prob- lems in both mathematics and physics. It is designed to push the boundaries of AGI search and often includes multimodal elements, requiring models to interpret diagrams and perform complex, creative reasoning.
B.3 MORE EVALUATION DETAILS
For evaluation, we adopt the SC@21 setting with a sampling temperature 0.7. Specifically, responses are generated, and after discarding cases where the answer cannot be extracted, the final
16

---

Published as a conference paper at ICLR 2026
prediction is determined by majority voting. In the event of tie, of the tied candidates is selected uniformly at random.
B.4
TRAINING PROMPT EXAMPLE
Completion Prompt
You helpful assistant that solves multiple-choice math questions with step-by-step reasoning.
System:
Please solve the following question carefully. Explain your reasoning, and conclude with the final answer using the format: Aboxed (x), where A, B, C,
User:
Example:
Question: What is 2 3?
A.
B.
D.7
Answer: = which option B. The final answer is Vboxed (B).
Question: fspecific math problem}
(option }
(option B
C. (option C)
D. (option D)
17

---

Published as a conference paper at ICLR 2026
C ANALYSIS
C.I PPO-BASED VARIANT: CAMMIIZER-ANOTICC INSTANTIATION OF CORE
The CORE framework is designed as an architectural module that operates independently of the underlying reinforcement learning optimizer. While the primary experiments utilize GRPO for ef- ficiency, CORE can be instantiated with other policy gradient methods, such as PPO, provided the structural requirement of multi-sampling is met. This section provides the technical details of the PPO-based implementation and corresponding performance.
ALGORITHMICC FORMULATION
To implement CORE within PPO backbone, batch configuration with N 4 trajectories per prompt is utilized. To isolate the effects of the conceptual intervention and reduce reliance on additional neural components, the standard critic network omitted. Instead, a group-wise Monte- Carlo baseline is employed for variance reduction. For each trajectory i, the discounted returns G; are first computed as follows:
T
G
t'=t
The advantage A; is then estimated through group normalization of these returns within the sampled group:
A; G; Egroup G
std group (G)
This formulation facilitates the integration of the CORE intervention logic, which encompasses both the Concept Bonus and Trajectory Replacement operations. Furthermore, such integration is achieved within the PPO clipped objective without necessitating any modifications the optimization step.
EXPERIMENTAL RESULTS
As summarized in Table 6 the PPO-based instantiation of CORE achieves consistent performance gains across most mathematical benchmarks. These results highlight the modularity of the frame- work: since CORE reshapes the training distribution and reward signals at the data-procsssing level, it functions effectively as higher-level supervision layer that is agnostic the specific choice of the RL optimizer.
Benchmark
Vanilla
CORE-CR (PPO)
Improvement
GK
55.3
$7.7
+2.4%
MM
29.8
30.1
+0.3%
AM
37.5
40,0
+2.5%
CM
37.3
43.2
+5,9%
MS
72.9
74.1
+1.2%
MW
96.8
97.5
+0.7%
OL
28.7
29.3
+0.6%
AD
95.1
95.2
+0.1%
MII
69.1
+0.1%
Table 6: Performance of the PPO-based variant under the SC@21 setting. Abbrevia- tions: AD=ASDiv, GK=Gaokao 2023 (EN), GS=GSM8K. MH-MATH. MM=Minerva math, MS-MMLU-STEM, CM-College.math AM=Amc23, MW=MAWPS, OL=OlympiadBench.
C.2 DISENTANGLING THE SOURCE OF CORE'S PERFORMANCE GAINS
To further disentangle the source of the performance gains achieved by CORE, conduct series of controlled ablation studies aimed at ruling out alternative explanations beyond our -nddet training mechanism. In particular, test competing hypotheses:
18

---

Published as a conference paper at ICLR 2026
Table Ablation on Qwen2-Math-7B under SC@21 (T=0.77. We compare: Vanilla; CORE-Base; Random- Reward GRPO; Top 4 of GRPO; and CORE-CR, Metrics are accuracy (%). Abbreviations: AD=ASDIV, GK=GAOKAO 2023 (EN), GS=GSM8K, MW=MAWPS, OL-OLYMPIADBENCH,
SV=SVAMP, TM-TABMWP.
Model
Method
AD
GK
GS
MW
OL
SV
TM
Vanilla
95.1
55.3
89.8
96.8
28.7
92.5
90.2
CORE-Buse
95.4
59.5
90.8
97.2
33.9
92.1
92.6
Qwen2-Math-7B
Random- Reward GRPO
95.2
55.8
89.9
97.2
29.9
91.2
89.6
Top 4 of GRPO
95.0
54.5
89.3
97.3
32.1
91.5
88.3
CORE-CR
95.7
58.4
91.1
97.3
34.5
92.7
93.6
H1: the gains are artifacts of the underlying GRPO optimization procedure.
H2: the gains primarily arise from verifier-based process supervision.
Below, we evaluate each hypothesis through targeted control experiments.
Testing H1: Do the Gains Come from CORE Rather than GRPO? Next, we investigate whether the observed gains are simply artifacts of the GRPO optimization procedure, rather than a conse- quence of the concept-guided currieulum introduced by CORE. To this end, we perform controlled ablations on Qwen2-Math-7BB holding compute, data, and hyperparameters fixed, and evaluate all variants under the same SC@21 (0=0.7) protocol.
Random-Rewald GRPO. For each generated response, we assign a binary reward at random (cor- rect/incorrect) and train with GRPO on these randomized labels. This tests whether stochastic re- ward noise alone can yield apparent performance improvements.
More-Candidatee Top-k Control. Standard GRPO samples four responses per input. Here we additionally generate two extra candidates (six in total) and then select the four highest-reward responses for training. This controls for the effect of a larger rollout budget and stronger within- GRPO selection pressure, without introducing any concept-guidn guided intervention.
Results. Table shows that Random Reward GRPO does not yield meaningful gains under the same setup. The Top 4 of 6 variant produces only small and unstable changes, often mild regres- sions. This indicates that larger rollouts or stricter in-GRPO selection alone do not account for the improvements. By contrast, both FORE-Base and CORE- CR consistently outperform the vanilla baseline across benchmarks, with CORE-CR CR achieving the best overall performance.
Testing H2: Process Supervision and Verifier-Guided RL. As first control, we compare CORE against a verifier-guided reinforcement learning variant, termed Process Supervision and Verifier- Guided RL and denoted as CORE-Base + Verifier. This baseline designed to contrast explicit concept- concept-guided intervention with passive process-level supervision, where learning signals are con- ditioned on intermediate reasoning trajectories rather than solely on final outcomes.
Concretely, Concept-Veringe Verifier is integrated into the CORE-Base training loop, using the same Qwen2-Math-7B model the verifier. For each generated reasoning trajectory, the verifier checks whether the target concept is explicitly and correctly invoked in the reasoning process. If the con- cept is correctly applied, an intrinsic process reward of +0.4 is assigned, independent answer correctness. This verifier therefore serves sparse process-level reward model targeting conceptual alignment. All other hyperparameters are kept identical to those used in CORE.
the final
Table 8 summarizes the performance of this verifier-guided RL baseline. While CORE-Base Veri- fier yiclds modest gains over the vanilla model on several benchmarks, it consistently underperforms CORE-CR. This suggests that explicit concept-guides intervention during training is more effective than verifier-based process supervision alone, and that simply rewarding the presence of correct concepts in intermediate trajectories is insufficient induce robust conceptual reasoning.
Overall, both control experiments consistently reject HI and H2. Neither verifier-based process supervision GRPO artifacts alone account for the observed improvements. The gains there-
19

---

Published as a conference paper at ICLR 2026
Table 8: Comparison with a verifier-guided RL baseline under SC@21 (T--.7). Columns use two-letter abbreviations: GS=GSM8K, AD=ASDiv, MW=MAWPS, TM-TabMWP, MH-MATH,
MS=MMLU-STEM, OL=OlympiadBench.
Model
Method
GS
AD
MW
TM
MH
MS
OL
Vanilla
89.8
95.1
96.8
90.2
69.1
72.9
28.7
Qwen2-Math7B
CORE-Base Verifier
90.8
94.5
96.0
92.5
69.7
72.9
34.2
CORE-CR
91.1
95.7
97.3
93.6
71.4
72.6
34.5
fore primarily stem from the concept-guided curriculum and intervention mechanisms introduced by CORE.
20

---

Published as a conference paper at ICLR 2026
C.3 EXAMPLE FOR ANALYSIS
Below an exmaple for analysis.
CORE-Base
CORE-Base
CORE-KL
Imput: Let A, B orthogonal matrices, & be nonzero real num- ber, and P an invertible matrix, Which of the following is true?
Input:
Input: Let A. B orthogonal matrices, & be nonzero real num- ber, and P an invertible matrix.
Concepts:
Orthogonal and Unitary Transfor-
mations:
Orthogonal and Unitary
Which of the following is true? A) + B also an orthogonal
A) + B also an orthogonal matrix
Matrices
An n-dimensional real matrix is called an orthogonal matrix if A' A = AA' I
matrix
B) kA is also an orthogonal ma- trix
B) kA is also an orthogonal ma-
trix
C) AB is also orthogonal
C) AB is also orthogonal
An re-dimensional complex matrix U is called unitary matrix if U = UT' L..
trix
trix
D) P-1AE AP is also an orthogonal
D) P- AP is also an orthogonal matrix
matrix
Let A, B be orthogonal matrices, he a nonzero real num- ber, and P an invertible matrix. Which of the following is true?
Exercise:
A) A + B is also an orthogonal matrix
B) kAA also an orthogonal ma-
trix
C) AB is also an orthogonal ma- trix
D) P-1AP AP also an orthogonal
matrix
Output: Answer: C Response: B
Output: Answer: C Response:
Output: Answer: C Response:
B
C
B) RA is also an orthogo- nal matrix. This is true. If A is an orthogonal matrix and & is nonzero real number, then A is also an orthogonal matrix. This is because multiplying an orthogonal matrix by a scalar does not change its orthogonal- ity. C) AB is also orthogonal matrix. This true. are orthogonal matrices, then their product AB is also an orthogonal matrix. This is because the prod- uct of two orthogonal matrices is orthogonal.... The final answer is (B).
B) kA also an orthogo- nal matrix This is true. If A is an orthogonal matrix, then &A is also orthogonal matrix for any nonzero real number k. C) AB is also an orthogonal matrix This is true. If A and B are orthogonal matrices, then AB is also orthogonal matrix. There- fore, the correct answer B) kA is also an orthogonal matrix.
B) KA not necessarily an or- thogonal matrix. If & is nonzero real number, then K is not or- thegonal unless & pmm1 This is because the columns of A are not necessarily orthogonal unit vectors. C) AB is an orthogonal matrix. The product of two or- thogonal matrices is also orthog- onal. This is because the columns of AB are linear combinations of the columns of A, which are or- thogonal unit vectors. Therefore. the columns of AB thogonal unit vectors. ... The cor- rect answer is C) AB is also an or- thogonal matrix. The final answer is boxed(C).
and B
also
Table 9: This is a case study comparing the original GRPO model, a variant augmented with concept prompts, and the model after CORE-KL training. The example illustrates that the original model tends to fall into a fixed reasoning pattern when dealing with orthogonal matrices. Even when provided with detailed conceptual derivations, the model still prefers rely patterns learned during pretraining. In contrast, the model trained with CORE-KL is able to break out of this fixed paradigm and effectively apply the relevant theorems of orthogonal matrices.
21

---

Published as a conference paper at ICLR 2026
D USE OF LLM
We have only used LLM for language polishing purposes in the paper writing. We do not use LLM for idea formalization, to extent that it could be regarded contributor.
22


Reviews:
[{"review_id": "Ex5TGoyPSL", "reviewer": "Reviewer_kr6a", "summary": "This paper focuses on the definition–application gap in large language models (LLMs) for mathematical reasoning, ie, models can accurately recite formal definitions but often fail to select and apply the correct concepts when solving problems.\n\nTo address this issue, the authors curate a structured concept–exercise corpus from a linear algebra textbook, containing 236 concept definitions, 703 examples, and 140 exercises. They further construct a Concept Probes dataset of 1,110 multiple-choice quizzes by generating items with Qwen2.5-72B and filtering them through GPT-4o for quality assurance. This dataset serves both diagnostic and training purposes.\n\nBuilding on GRPO reinforcement learning, the authors propose CORE (Concept-Oriented REinforcement) through three variants:\n\n1. CORE-Base: standard RL training directly on the concept-probe dataset.\n\n2. CORE-CR: when all sampled trajectories fail, failed ones are partially replaced with concept-guided rollouts and assigned a reward bonus.\n\n3. CORE-KL: upon failure, the model minimizes the forward KL divergence between the concept-guided and unguided policies, effectively distilling the reasoning distribution that emerges when a concept is provided.\n\nExperiments on Qwen2-Math-7B and Llama-3-8B-Instruct show that all CORE variants consistently improve both in-domain textbook exercises and multiple out-of-domain mathematical reasoning benchmarks, demonstrating enhanced concept selection and application.", "questions": "See Weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces concept texts as a controllable and fine-grained supervision signal in reinforcement learning, offering richer and more directed feedback compared to traditional scalar rewards based solely on final answer correctness.\n\n\nExperiments are conducted on both base and instruction-tuned models across multiple, stylistically diverse mathematical benchmarks, demonstrating the generality and robustness of the proposed approach.\n\n\nThrough small-scale manual case analysis and perturbation tests with irrelevant concepts, the authors provide preliminary evidence that the performance gains stem from improved concept selection and application rather than superficial heuristic matching.", "weaknesses": "The comparison with relevant baselines is insufficient: the paper does not include fair experimental comparisons with conceptually similar methods such as BREAD (which also employs failure-branch replacement), nor with other related process-supervision or verifier-guided RL approaches. As a result, the marginal contribution and uniqueness of CORE remain unclear.\n\nMost reported performance improvements are relatively small and lack statistical significance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rXXLKd0x3x", "reviewer": "Reviewer_vWn1", "summary": "This paper presents concept-oriented approach to create synthetic data based on math concepts. The initial diagnostic evaluation results show that base models, while perform well in standard evaluation, they are prone to error when tested with variation inputs (that permutes parameters). The paper then presents an RL based approach to train models based on guidance from concepts in events of reasoning divergence. The results show that Core RL approaches can generalize beyond the synthetic puzzles.", "questions": "Please elaborate comparison of three RL approaches, and how they differs in contribution to performance gains. It's good to understand importance of augmented data vs new RL methods.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- puzzle dataset construction based on math text book\n- diagnostic analysis about issues of why current models could fail in truly understand math concepts.\n- RL frameworks to train models to be coherent with concepts with different reward models\n- evaluation shows out of domain generalization", "weaknesses": "- difference in RL methods CR / KL have some improvement over the baseline RL-base, it is not clear how these methods truly improve on top of baseline RL approach.\n- (neutral comment) another simple baseline of enhancing models with variation data (e.g., with permuted params) could also be used to understand the wether concept reasoning is important, or it is just a matter of data augmentation is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "747I4zawId", "reviewer": "Reviewer_6tm1", "summary": "This paper introduces CORE, an RL training framework that turns explicit math concepts into a controllable supervision signal. The proposed approach comprises three key components: dataset curation, gap diagnostics, and a concept reinforcement recipe. The framework is evaluated on two 7B models across multiple mathematical reasoning benchmarks, demonstrating clear effectiveness under the given experimental setup.", "questions": "See the weaknesses section", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper targets an important research problem: LLMs often solve math problems by pattern matching rather than genuine conceptual understanding\n\n2. The proposed approach, including the curated dataset and the CORE-KL framework, demonstrates clear value and effectiveness.", "weaknesses": "1. My main concern lies in the experimental setup. Although the proposed method shows effectiveness under the reported setting, the evaluations are primarily conducted on relatively simple or medium-difficulty math benchmarks. Moreover, the baseline results (Table 2) are inconsistent with those reported in the corresponding technical reports. For example, the performance of Qwen2-Math-7B on GSM8K and MATH does not match, and it is unclear why the model performs significantly worse on the simpler GSM8K compared to the more challenging MATH.\n\n2. The RL training set is synthesized and evaluated by LLMs, raising concerns about data quality and reliability. It remains unclear how the authors ensure the correctness of the training data and mitigate potential issues such as reward hacking when using LLMs as judges.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fPBwmRf3Wc", "reviewer": "Reviewer_b4Hh", "summary": "This paper proposed the make a new math dataset that, in addition to question and answer pairs, augments related concepts by extracting and translating content from a Chinese math book. The authors then compared 3 GRPO based methods to show the effectiveness of their proposal.\n\nAlthough the paper claims in the abstract that CORE is \"an algorithm-agnostic training framework that turns explicit concepts ...\", the fact that its algorithm relies on the $N$ generation failures suggests it is a derivation of GRPO, which invalidates the claim. In addition, I have 2 major concerns.", "questions": "See details in the weaknesses section.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper is tackling an important problem: \n* While some details are hard to get (e.g., Sec 3.4), the paper is generally well-written.", "weaknesses": "* Unclear technical novelty of the RL method vs. the training data. While the paper emphasizes the goal of bridging the 'definition-application gap', its primary technical contribution (core-cr and core-kl) appears to be an application of existing RL strategies rather than a novel method. The core mechanism involves dynamically intervening with expert guidance when the policy fails. This is methodologically similar to prior work on \"teacher\" or \"expert-anchored\" RL. The paper's novelty therefore seems to be rest on the content of the intervention rather than the mechanism. But are the performance gains attributable to the unique properties of using a declarative concept, or would any high-quality reasoning trace produce the same benefit when used within this dynamic-intervention framework? The paper lacks an ablation study to disentangle the value of the content from the value of the method.\n* Generalizability and conflation of methodological contribution with data curation. The method section describes a non-trivial, multi-stage curation pipeline to create the data, involving generations by Qwen2.5-72B-Instruct, vadilation and filtering by GPT4o, cross-model validation strategy, etc. This rigorous process suggests that the effectiveness of the proposed framework is linked to the quality fo the dataset. Actually, the core-base model, trained on this data, already shows significant improvement over the baselines. When comparing the performance boost after applying core-cr and core-kl, only TB and CM showed meaningful improvements. This thus questions if a large portion of the gains come from the data itself rather than the specific CORE-CR or CORE-KL intervention recipes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)