Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?
Authors: Chuxuan Hu
Yuxuan Zhu
Antony Kellermann
Caleb Biddulph
Suppakit Waiwitlikhit
Jason Benn
Daniel Kang
Venue: ICLR 2026 Poster (2026)
Keywords: large language models
reinforcement learning
supervised fine-tuning
generalizability
Primary area: reinforcement learning
TL;DR: In this paper, we systematically investigate the generalizability of reinforcement post training.

Abstract: Reinforcement post training (RPT) has recently shown promise in improving the reasoning abilities of large language models (LLMs).
However, it remains unclear how well these improvements generalize to new domains, as prior work evaluates RPT models on data from the same domains used for post-training. To understand the generalizability of RPT, we conduct two studies with specific focus on Reinforcement Learning with Verifiable Rewards (RLVR). (1) Observational: we compare a wide range of open-weight RPT models against their corresponding base models across multiple domains, including both seen and unseen domains in their fine-tuning data. (2) Interventional: we fine-tune LLMs with RPT on single domains and evaluate their performance across multiple domains. Both studies converge on the same conclusion that, although RPT brings substantial gains on tasks similar to the fine-tuning data, the gains generalize inconsistently and can vanish on domains with different reasoning patterns.

Published as a conference paper at ICLR 2026
BREAKING BARRIERS: Do REINFORCEMENT POST TRAINING GAINS TRANSFER TO UNSEEN DOMAINS?
Chuxuan Hu', Yuxuan Zhu', Antony Kellermann, Caleb Biddulph,
Suppakit Waiwitlikhit, Jason Benn, Daniel Kang' Siebel School of Computing and Data Science University of Illinois at Urbana-Champaign (chuxuan3, yxx404, ddkang) @ fillincis edu
ABSTRACT
Reinforcement post training (RPT) has recently shown promise in improving the reasoning abilities of large language models (LLMs). However, remains unclear how well these improvements generalize to new domains, prior work evaluates RPT models on data from the same domains used for post-training. To understand the generalizability of RPT, conduct two studies with specific focus on Reinforcement Learning with Verifiable Rewards (RLVR). (1) Observational: we compare a wide range of open-weight RPT models against their corresponding base models across multiple domains, including both seen and unseen domains in their fine-tuming finettuing data. (2) Interventional: we fine-tune LLMs with RPT on single domains and evaluate their performance across multiple domains. Both studies converge on the same conclusion that, although RPT brings substantial gains on tasks similar to the fine-tuning data, the gains generalize inconsistently and can vanish on domains with different reasoning patterns.
INTRODUCTION
Large language models (LLMs) demonstrate strong capabilities across diverse reasoning settings. In mathematics and quantitative reasoning, recent models achieve near-expert performance on benchmarks including GSM8K, MATH, and AIME (Cobbe et al. 2021 Lightman et al. 2023
Jia 2024 Tunstall & Jia 2024). In code generation and program synthesis, L.MMs similarly show
substantial progress across challenging evaluations (Austin et al. 2021 Chen et al.) 2021 Jain
ctt al. 2024 Shi et al. 2024 Zhuo et al. 2025 MatrixStudio 2024| Aider-Al 2025). ). Beyond
reasoning spanning legal, financial, and biomedical tasks (Guha et al. 2023 Zhang et al. Jin et al. 2019 Griot et al. 2025). A growing body of work shows that reinforcement post-training (RPT) yield large performance gains on these tasks. Recent RPT-enhanced models achieve dramatic improvements in competition-level mathematics and coding benchmarks, in
these structured domains, LLMs also perform well on knowledd wwwoledgeinntnstiie
cases matching or exceeding top
human competitors (Shao et al. 2024 Luo ct al. 2025a b Su ct al. 2025b Zhao et al. 2025a Yuan
al. 2024 Cui et al. 2025 Xie
2025 Team 2025a He et al. 2025 Liu
2025 Google
Mind 2025a
XA] 2025 Granite Embedding Team 2024 OpenAl et al. 2024; Anthropic
2025p. This raises an important question: does RPT provide generalizable improvements, as broadly as those achieved through pretraining?
Existing evaluation frameworks and RPT setups provide limited evidence answer this question. To address it systematically, we design two-stage investigation pipeline (Figure Tab. with a specific focus on Reinforcement Learning with Verifiable Rewards (RLVR), the dominant instantiation of RPT for verifiable reasoning tasks.
First, prior work evaluates RPT models within their post-training domains (Luo et al. 2025b a). To overcome this limitation, we conduct an observational study in which we evaluate 18 recent opegh RPT models with publicly disclosed post-training data alongside their corresponding base models across wide range of domains, including legal. financial, and medical benchmarks, spanning their seen and unseen domains. This study designed to provide an initial view into the generalizability of RPT.

---

Published as a conference paper at ICLR 2026
Structured Reasoning
Observational
Study
Math
Math-RPT Model
Math
Code
Code
Code-RPT Model
Open-sourced Models RPT
Disclosed Data
Knowledge
Knowledge-RPI Model
sntoweddgeInneenstee Reasoning
Self-curated
Legal
Finance
Medical
Domain Data
Interventional Study
Mutt-Domain Evaluations
Generalizable
Generatzzblle
(a) Overview of our two-stage investigation pipeline.
(b) Summary of RPT generalizability.
Figure 1: The method (a) and key findings (b) work. Through unified multi-domain evaluation framework combining observational and interventional studies, we find that RPT exhibits limited generalizability across domains.
Additionally, we notice that these RPT models, as representative selection of existing open- weight models such as DeepSeck RI (DeepSeck-AL) et al. 2025). are post-trainee -isi-trained on mixed domain data. The presence of such confounding factors makes it difficult to isolate and interpret the generalizability of RPT at a finer granularity. To strengthen our findings, we conduct an interventional study in which post-train LLMs via RPT math, coding, and knowledge-intenn inttegivveeeeeeeeeeeee reasoning data and evaluate their performance on both in-domain and out-of-domain tasks. We illustrate our methodology in detail in Section 3
As we summarize in Figure Tb our findings show that gains from RPT shows limited generalizabilty. Specifically, RPT gains domains involving structured reasoning patterns (e.g., math, code) gen- cralize well within and across structured domains, but fail to generalize to unstructured domains. In contrast, gains from RPT on unstructured domains (c.g., legal, finance) do not generalize well within unstructured domains, but show transferability to structured domains. We analyze these results comprehensively in Section
Our findings suggest that RPT models are most effective when the target task shares similar reasoning patterns with RPT data. Consequently, while RPT remains a powerful method for improving LLMSs performance, its benefits are largely limited to the domains represented in the post-training data and do not generalize to a wide range of new, unseen domains.
BACKGROUND
In this section, introduce the motivation of our study. We begin by demonstrating the strong performance of LLMs post-trained via RPT across various reasoning tasks, particularly mathematics and coding. We then diseuss the limitations of existing work in understanding the mechanisms and boundaries of RPT. We close by introducing the data-domain taxonomy that grounds our study.
RPT models demonstrate promising performance across a wide range of tasks. RPT models have achieved remarkable improvements on complex reasoning benchmarks, For example, Gemini Pro (Google Mied 2025b) achieves 100% accuracy on AIME 2025 (OpenCompass Community 2025), math competition benchmark. Claude Opus 4.5 (Anthropic (025) reaches 81% accuracy on SWE-bench Verified (Jimenez et al.] (024), a coding benchmark, while GPT-5.2-PPrs (OpenAI 2025) reaches around 93.2% accuracy on GPQA Diamond (Rein al. 2023). a benchmark for graduate-level scientific reasoning.
Out-of-domain generalizability of RPT models remains understudied. Despite impressive results, recent work has examined the limitations of RPT models and the opaque nature of their underlying reasoning capabilities (Team 2025b Yue et al. 2025 Ma et al. 2025 Ye et al. 2025)
2

---

Published as a conference paper at ICLR 2026
In particular, there is growing interest in the role of RPT data, especially the extent to which RPT algorithms rely on large, diverse corpora to achieve generalization (Zhao et al. (025a).
While RPT models benefit from training diverse. mixed-domain data, this diversity makes it difficult to directly assess their generalizability to unseen domains. As a result, prior work evaluates RPT models tasks within the same domains their training data (Team] 2025b Yue al., [025]
Ma et al. 2025 Ye et al. 2025). However, many reasoning tasks remain underrepresented or entirely absent from existing training corpora. Exploring the generalizability of LLMs trained with RPT to
such tasks is therefore essential for identifying the boundaries of their applicability in real-world, complex scenarios.
Understanding RPT generalizability requires systematic view of reasoning domains. Fol- lowing the task suites defined in prior work (Su et al. (025a)), we focus on three major domains of interest: code, math, and knowledge-inteniiee reasoning (Figure lb). [I). These domains are chosen to capture a broad spectrum of reasoning challenges commonly seen in language model evaluations. The knowledge-intensive reasoning domain can be further divided into application-speciffe subdomains such legal, finance, medical, etc. We not only examine performance across the high-level domains, but also evaluate how reasoning patterns and generalization behaviors vary across subdomains.
Among the three, consider math and code domain tasks to follow structured reasoning patterns, where solutions follow deterministic logical steps and require precise syntax and formal semantics (Su et al. 2025a). In contrast, tasks within the knowledge-intensifn reasoning domain require more flexible and context-sensitive reasoning, referred to here as unstructured reasoning. We define unstructured reasoning as problem-solving processes that do not adhere to fixed sequence of logical operations and lack well-defined intermediate representation or symbolic grounding. Such tasks demand broader world knowledge, interpretive judgment, and the ability to handle ambiguity or incomplete information. For instance, legal and financial question answering may involve interpreting lengthy documents, extracting relevant information from loosely connected statements, or evaluating conflicting evidence. We quantify the rationale of such divisions in Appendix B
STUDY DESIGN
We present our study design that aims to investigate the generalizability of RPT. We propose the following research questions (RQs) that have not been systematically examined in prior work.
(RQ1) Cross-domain generalization. To what extent do the capabilities acquired through RPT transfer to tasks from domains not included in the training data?
(RQ2) Role of reasoning structure. How does the structure of reasoning required by a task affect generalization? Do skills learned from highly structured domains (e.g., mathematics, code generation) transfer to less-structured domains (e.g., medical or legal reasoning), and vice versa?
(RQ3) Intra-domain generalization. How effectively do RPT gains generalize across subdomains within the same domain?
(RQ4) Stability of generalization. the generalizability RPT consistent across different hyper- parameters, such as model size, training algorithm, and the number of RPT steps?
To address these research questions. we design two-stage pipeline. First, we perform an obser- vational study by evaluating 18 RPT models, each compared against its corresponding base model across diverse set of benchmarks, spanning their seen and unseen domains. Because existing RPT models are typically trained with different configurations (e.g., different RL algorithms and hyperparameters) multi-domain data, it is challenging to isolate the effect of RPT itself from the advantages brought by specific configuration or data.
To mitigate confounding factors, we further conduct an interventional study, where we post-trained three RPT models from the same base model with the same configuration, each on a disjoint single-domain dataset. We then evaluate these trained models using the same benchmarks. as in the observational analysis. In the rest of this section, we describe our evaluation settings and experiment setups for both studies.
3

---

Published as a conference paper at ICLR 2026
3.1 EVALUATION SETTINGS
Benchmarks. For evaluation, we 16 popular benchmarks, covering a wide range of domains and difficulty levels. We categorize these benchmarks into the following three representative domains:
Math: For easier questions, we use GSM8K (Cobbe et al. 2021) and MATH-500 (Lightman et al.
2023), while for more challenging problems, select AIME 2024 Jia 2024 and AMC 2023
(Tunstall Jia 2024).
Code: We use easy coding problems, including MBPP (Austin al. 2021 and HumanEval
(Chen et al. ( and relatively challenging problems, including Big Bencchh (Zhuo et al.
2025).
(Jain et al. 2024). USACO (Shi et al.) 2024), and Codeforces Studro 2024 To test programming language generalization, we also include the multi-language benchmark
LiveCode
Polyglot AA 2025).
Knowledge-intensive reasoning: We use high-quality benchmarks that are not mathematics nor programming problems for knowledge-intensive reasoning, including PubMedQA (Jin ct al. 2019) and MedQA (Griot
2025) for medical reasoning, TabFact (Chen et al.] 2019) for fact verification, LegalBench (Guha et al. (023) for legal reasoning. and FinBench (Zhang et al.) for financial problem solving.
Generation Configurations. For all benchmarks, we use a consistent set of generation hyperparam- eters across all models. The maximum response length is min(16192, C}, where C' denotes the model's 's context window. For each model, we run the small benchmarks (i.e., AMC 2023 and AIME 2024) 16 times, while executing all other benchmarks once. For prompting, we apply each model's 's default chat template and system prompt.
Evaluation Metrics. To assess whether RPT improves the accuracy performance within or across
domains, we report the aggregated accuracy improvement A (P) of an RPT model over its base model j for a given domain D:
EIED N, -..4.-AA44.. -A,))
(D) tjj
EIED N,R.
where N₂ is the number of problems in t, R₁ is the number of repetitions we executed for f, Ai, is the accuracy of model on benchmark and Aj, is the accuracy of model j benchmark t.
In addition, to ensure statistical significance in our findings, we applied the Cochran- (CMH) test (Agresti 2013), statistical test for analyzing stratified categorical data. We treat each benchmark as an independent stratum-the is, a random sample of distinct downstream tasks. Given an RPT model base model j, and domain D of benchmarks, calculate the common odds ratio estimate (Q..g.) that estimates the correlation between the RPT process and the accuracy improvement on D;
EIED (1 - Aj,))
EIED N,R,A,,, A,,))
An odds ratio greater than indicates improvement from RPT; value less than I indicates a decrease accuracy from RPT. evaluate the statistical significance under the null hypothesis Ho o = against the alternative hypothesis H1
0DD) A 1, using the standard CHM test statistics.
EEED (A - AA,,)
< EIED NE A,, A
(2N,
which follows chi-squared distribution asymptotically with 1 degree of freedom. For all reported values, an asterisk (*) denotes statistical significance at p 0.05.
3.2 EXPERIMENTAL. SETTINGS
Observational Study. To ensure a comprehensive and representative evaluation of RPT model
generalizability, adopt systematic approach to selecting models for our observational study:
4

---

Published as a conference paper at ICLR 2026
Table 1: Selected RFT models for observational analysis.
The RFT Domain (s) refers to the
domain(s) covered in the RFT training data.
(Model (D) RPT Model
Base Model
RPT Domainis)
(2) DeepScaleR 1.3 Preview
Luo
20250
DeepSeck 81 Distil Qwen (DegSSekk Al ct
205
Muth
(2) Chdpppoae--11------ P.s-Proci
DeagSeck-R1 1i-------lll--eene 5B (ASIpN of
Mlx
Code
(3) -oyyoottol---pe---laa Inno- Tra u1 Ream 2024
Llainn-11-88--oouuu MMea All brvee
Code, Math
(4) Eunus-2-7B- TR-PRIME et
2023
Yaaner 2016
Fumu-2-7B--SF7 (or 1024 2004 Yoond el a 204
Coda, Math
(3) Absolute Zero Reastter Coakr 39
20243
Qwes2.5 Coder 3B
[Eur e Di 2023
at 2024
Code
Absolute Zeto Reavens-Coaar- 7ile
(PDr (unle-5-
20133 Tram 2015
Colc
11/211-.55 1.8 (Zyphmn 20124]
DeepSeck-81-Diiiie Dill-DDi--/scunne
205
Cole, Math
Lama- Neterrreettone-
(Benowich al. 2025
Llmm-331--0--me [Meta
Instructio Fullowing
Thespis Lluma-3.1-88
[Locuusque
Meta-Lluma-31 Intrurr--nneeraa
Clai
STI 1TT1.1-3-1.5--11- -3-1.58-pm
DecySack-R1
Math
DecgSock-R1 Bat-:al--R---iii---eeeeeeee
Math
(hunett
R05
Uaamo 3.-8-3..-80--mmommm: A1202
(13) OREAL 11L
OREAL-7B.SPP
Math
Mith
(15) Desppoooollll
Code
(16, OREAL-3233 (ha at
et
Para
ORCAL-320-STT Lasr 2003
Mach
2023
Qwes3-14B [Jeum 20A00
Finance
(18/Absoluiz_44 Zelo Reuooie Couer 14b Zhao ct 20250
Qwee2.5 Cod2r 1ob yrurrs
2012 Ying etal. 2044
Cods
Stage I. We collect the 466 models from Hugging Face applying the following filtering criteria: as of April 23rd, 2025: (1) the model supports Text Generat on tasks, (2) model card description contains the keyword reasoning, chain-of-thowe (3) the model has received at least likes.
and/or chain of thought and
Stage 2. We use o4-mini OPeenAl 2025) to prefitter models potentially trained with RPT, based on their model card descriptions. This automatic filtering is followed by manual verification, resulting in 31 models that we confirm to be RPT models.
Stage 3. From the 31 RPT models, manually select 12 that meet the following criteria: (1) the RPT datasets are publicly disclosed, (2) the model sizes range from 1.5B to 8B parameters, and (3) the base models are not purely pretrained models, ensuring they can generate coherent responses and follow basic instructions for evaluating reasoning capabilities.
We evaluate Absolute Zero Reasoner-Coder- demonstrating strong performance on math and code reasoning tasks. We view it as a representative case for examining RPT generalizability. We also evaluate the variants of all selected RPT models with more parameters, whenever such variants are available.
post-trained with limited RPT data but
We finalize our selection of 18 RPT models, with the details, including base models and RPT domains, presented in Table For each RPT model and its corresponding base model, we compare performance across 16 benchmarks.
Interventional Study. To isolate the effect of RPT from other training configurations, in- cluding datasets, algorithmss, and hyperparameters, DeepSeek- -Di--Ditilll----
trained three RPT models based 5B (DeepSeek-AT et al.) 2025) on three disjoint datasets, math, code, and knowledge-intensive reasoning, respectively. We curated training datasets based on existing datasets that led to performant RPT models and cleaned them to ensure that the training datasets do not overlap with our evaluation sets. We built the following datasets:
Math: uniformly sampled 40,000 problems from combination of the math split of Eurus-2-RL
(Cui et al. 2025), which originates from the NuminaMath-Cof dataset (Ll et al. 2024).
Code: uniformly sampled 40,000 deduplicated problems from combination of KodCode (Xu
et al.
2025), Dree
Preview (Luo et al. 2025a). Apps (Hendrycks et al., 2021), TACO
al.
2023 and the code split of Eurus-2-RI RT. (Cui et al. 2025).
Snowledge-intensiie Reasoning: we selected 40,000 high-quality, non-math, and non-code data from the multi-subject RPT dataset (Su et al., 2025b). To achieve that, applied 03-mini (OpenAl 2025) to exclude math-related, code-related, or fact-recall questions.
We applied consistent settings for all three RPT training processes. In terms of the RL algorithm. applied Group Relative Policy Optimization (GRPO) with the same setting DeepCoder (Luo ett al. 2025a) with the reward definitions and dynamics included in Appendix [D] In terms of hyperparameters, trained each dataset for one epoch with batch size of 64 and context length
5

---

Published as a conference paper at ICLR 2026
of 8.192. To stabilize the training process, used learning rate of 10-6 and entropy coefficient of O. We post-trained the models on 8 80GB H100 GPUs.
Furthermore, conducted three additional experiments under alternative settings, to validate our conclusions about the generalizability of RPT,
1. We trained with DAPO (uu et al. 2025). ttateoof-the-an RI algorithm.
We extended the training process to 2 epochs and evaluated intermediate checkpoints
3. We trained with L1 ama-3.
(Meta Al 2024a) the base model.
4 FINDINGS
In this section, we present the findings our study based on results from observational and interven- tional studies. We summarize findings as follows:
(RQ1) RPT does not exhibit generalizability in arbitrary unseen domains (Section 4.1
(RQ2) RPT demonstrates cross cross-domain generalizability when reasoning patterns similar, such as mutual transfer between math and code, but fails to generalize across distinct reasoning patterns, such as from math or code knowledge-intensive reasoning (Section 4.2
(RQ3) Intra-domain generalizability of RPT strongly depends on the structural similarity between subdomain tasks (Section 4.3).
(RQ4) The generalizability of RPT is consistent across different hyperparameters (Section 4.4)
4.1 RPT GAINS DO NOT GENERALIZE TO ARBITRARY UNSEEN DOMAINS
Existing RPT models fail to transfer beyond their training domains. We begin by analyzing our observational study, which evaluates a diverse set of existing RPT models using multi-domain tasks. Specifically,
compare the performance improvements of each model in tasks from the same domain as their training data (ID), and tasks that are out-of-domain with their training data (OOD). For instance, (I) DeepScaleR 1. S---reviee is trained exclusively math-related data. Therefore, ID tasks for this model include GSM8K, MATH500, AIME 2024, and AMC 2023, while all other tasks (e.g., legal, medical, coding) are OOD.
We present the results in Table Across the table. RPT models exhibit considerably higher improve- ments on ID tasks compared to OOD tasks, with 2.87% increase in pass@ I for ID tasks, but a 3. 1.99 decrease for OOD tasks. For example, (1) DeepScaleR- 1..5aleeR--.B--Peeeeee on math domain tasks, but only 1.7% in others, representing 3x drop. This lack of generalizability stems from the RPT algorithm itself rather than simply from overfitting to large-scale training data: notably, a similar trend observed in (6) Absolute_Zere Sertee_rrr_____e Saserr_Reasoeeeee post-trained
shows a 5.1 5.1% gain in pass @)
which was near-zero amount of data. Despite its minimal training data exposure, this model experiences a 23.31% % decrease in pass @ accuracy on unseen domains, while achieving a 30.12% improvement within its RPT domain. We also observe that the generalizability of RPT algorithms is sensitive to the training data, implementation details, and finetuning strategy. For example, although all trained on math data, (1) DeepecaleeRR11 5B Preview demonstrates improvements both ID and OOD tasks, whereas (7) ZR1 121-1.22. 1. 5B and (10) STILL- 3-1---1---0 5B Previes ew show statisti- cally significant performance gains in / ID tasks as well as statistically significant performance drops on OOD tasks. These findings suggest that the gains from RPT are largely Nomain-sspeiffi models significantly improve on tasks similar to their training data, but fail generalize robustly other unseen domains.
Single-domain finetuning reinforces evidence of RPT's limited generalizability. To further dissect the generalizability limitations identified in our observational study, we conduct a more controlled investigation by isolating models post-trained exclusively on single domains. To do so, analyze our interventional study results, where ID corresponds to the training domain, while OOD include all tasks from the remaining two domains in evaluation.
As shown in Figure 2 none of the models post- trained on single domain exhibit statistically sig- nificant improvement on OOD tasks. Both the Math-RPT model and the Code-RPT model show

---

Published as a conference paper at ICLR 2026
Table Existing Open-sourced RPT models achieve significantly larger accuracy gains A (%) and
odds ratios & on in-domain (ID) tasks compared to out-of- (OOD) tasks.
Metric
(2)
22]
15p
(4)
15y
(6)
171
780
191
100)
(23)
(22)
(22)
(24)
(13]
116)
(1%)
(1A)
Ang:
30.12
-25.01
-0.50
-1641
-3.00
M1
-341
1523
-..3
-1.33
-7.99
-27.80
40.94
-5.68
-943
A1000)
15332
53.43
6.15
3.12
22,47
133
0.85
0.86
0.41
0.50
0.58"
0.05"
0.697
0.907
0.72
0.70
127
2.75
54.61
1.33
0.68
1.35
1.00
2.74
0.98
1.22
Ir-oomann
DutmanD
In-oomann
Out-af-Demain
Mach-RPT
3:13
1,19
0.93*
1.38
Model
Knowledge-RP1 RPT
88*
Mocel
0.54
1.04
5
0.9
122
13
Passgi1 Improvement (%6
Cdds Ratio
Figure 2: RPT models on single domains show significant pass@1 improvements over base models and higher odds ratios on in-domain tasks, but not on out-of of-mnn tasks. No single-domain model achieves statistically significant gains in out-of-domain tasks.
performance drops on OOD tasks with statistical significance, in contrast the statistically signifi- cant gains they achieve in-domain. The Knowledge-Ril -PT model also demonstrates no statistically significant gains on its OOD tasks.
4.2 RPT GAINS GENERALIZE ACROSS DOMAINS WITH SIMILAR REASONING PATTERNS
suuuttured-to-stuucuur generalization is effective. We observe that models post-trained math and code data exhibit strong mutual generalizability. In our observational analysis (Figure models post-trained exclusively math code demonstrate transferable performance gains across these two domains. For example, models post-trained on math domain data achieve an average improvement of 281% in pass@1 on math domain tasks and 4.77% code domain tasks. Similarly, models post-trained on code domain data improve by 9.49% in pass @ on code domain tasks and 15.44% on math domain tasks. In both cases, the improvement even greater on the non-finetuned domain, suggesting that math and code tasks share common structured reasoning patterns that enable RPT to generalize effectively across these domains.
Structured reasoning patterns that are more foundational tend to exhibit stronger cross-domain transfer. Building on the findings from our observational study, we further examine the generalizabil- ity across structured reasoning domains using interventional study results from models post-trained on single domains (Figure 4 We observe that the generalizability from math to code is notably stronger and more consistent than the reverse. This aligns with the intuition that mathematical reasoning is a more fundamental form of structured thinking, serving the backbone for coding tasks, and thus enables better cross-domain transfer when used as RPT data. We provide a quantitative analysis of reasoning trace similarity across structured domains in Appendix B.I
Structured-to-un unnntuuuurrd generalization is limited. Models trained on structured reasoning domains, such as math and code, exhibit substantially reduced improvements when evaluated on -nwwledge-intensive domain tasks. In our observational study (Figure B.. models post-trained on structured reasoning domains (i.e., math, code, or both) achieve an average improvement of -0.27% in pass@ @1 on knowledge-intensiv domain tasks, compared to significant gains of 1 11.08% on math and 5.82% on code tasks. While the improvements in math and code domain tasks are statistically significant, the performance drops on knowledge- knowledge-intensiee intensive reasoning tasks, indicating a lack of generalizability unstructured domains. For instance, in the (I) DeepScaleR-1 5B-Prev reeriiiww and (2) DeepCoder both significantly higher than those in knowledge-intensivg domain tasks. Our interventional study
C8-Previees wiew pair, the observed gains in math and code domain tasks are
7

---

Published as a conference paper at ICLR 2026
math knowledge
1.65
3.93
-26.41
-3.70
2.62
-24.5
-.447
7.07
26.01
-1.38
-7.39
22.87
8.13
-0.32
-3700
0.15
4.71
24.89
40
10
code
2.32
4.61
1.47
4.30
-6.27
30.12
2.37
6.66
.8.78
0.16
-3.41
-3.20
17.43
-0.83
-1.87
-5.48
.8.81
25.01
(3)
[4]
(6)
(9)
(10)
11.11
(12)
0130
(14)
(15)
[16)
(17)
(101
(a) Pass improvement (D)
in percentage across domains.
math Knowledge
1.07
0.31
0.86
0.80*
0.71
1.01
101
1.36
1.58"
2.70
4.25*
0.96
7.62*
1.22*
2.99*
1.20"
0.60
0.81
0.97
0.97
1.12
0.88
0.87
0.49
1.06
300
code
0.60
(1)
(2)
(3)
(4)
(5)
(6)
17)
18)
19)
110)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(19)
(b) Odds ratio ((P) across domains.
Figure 3: Multi-domain evaluation results of existing RPT models. We highlight in-domain results with frames. RPT shows mutual generalizability between math and code, one-way transfer from knowledge-intensive reasoning to math and code, but no generalization from math code to knowledge-intensive reasoning.
Knowledge
-1.93
-3.98
-3.1
Knowledge
0.92*
0.96*
0.88
1.3
Code
1.7
4.21
-0.02
1.15*
1.38*
1.00
1.1
0.8
Math-RPT
Code-RPT
Knowledge-RPP RPT
Nath-RP1
Code-RP1
Krewledge-RPI
(a) @ improvement (P)
across domains.
(b) Odds ratio (PP) across domains.
Figure 4: Multi-domain evaluation results of single domain RPT models. We highlight RPT domains with frames. RPT demonstrates generalizability from math to code and from knowledge-intensivg reasoning math, but shows no generalizability from math code to iittengeve reasoning.
results further confirm this trend (Figure 4) while the Math-RPT model shows improvements in both math and code domain tasks, its performance drops notably on knowledge-intensive domain tasks. Similarly, the Code-RPT model shows statistically significant drop in performance knowledge-intensiee reasoning tasks. These results suggest that although structured reasoning skills generalize well across similarly structured domains, they fail transfer effectively domains that require less structured, more heterogeneous reasoning patterns.
sestrrutured--o-sttrtttrr generalization is promising, RPT models trained on unstructured poowledge-intensive domain data still exhibit measurable gains in structured tasks. our observa- tional study (Figure 3) nhowledge-intensie improvements on math (21.40%) and code (12.16%) tasks compared to tasks within the knowledge- intensive reasoning domain. Similarly, in our interventional analysis (Figure 4. the Knowledge-PP model achieves statistically significant gains on math domain tasks and shows no noticeable degrada- tion code domain tasks, while underperforming that unstructured reasoning patterns encompass broader representational complexity and implicitly subsume the essential components of structured reasoning, functioning conceptual superset.
domain RPT models show substantially higher pass @
tasks within its own domain. This suggests
8

---

Published as a conference paper at ICLR 2026
Table 3: RPT configuration variants consistently fail improve generalizability, as shown by accuracy gains A (%) and odds ratios 9 on in-domain (ID) tasks versus out-of-domaij (OOD) tasks across different base models and RPT algorithms.
Base Model RPT Algorithm
A
41000)
A(D)
((000))
(
g(o0D) f
( /8(000)
DeepSeck-RR MelSscck-R-Disslie RI--Disill-Qwen--te GRPO
3,13
-1.81
4.94
1.1881
0.9209
1.2812
Llama-3.2-333 3-- Instruct GRPO
6.47
1.41
5.06
1,4694"
1.0635'
1.3817
DeepSock-RI Diiii--RR--siill--e----..-.. DAPO
3,96
-1.27
5.23
1.2390'
0,9182
1,3070
4.3 INTRA-DOMAIN RPT GAINS DEPEND ON SUBDOMAIN STRUCTURAL. SIMILARITY
Structured reasoning patterns generalize effectively within domain. Consistent with prior work on math and code reasoning, our observational study shows that models post-trained on structured domains generalize well across tasks within the same domain (Figure 3 On average, models trained math data achieve a pass @1 improvement of 2.188% on math tasks, while models trained code data show an average improvement of 9.49% on code tasks. Our interventional analysis further confirms this trend where frructure-domain models (i.e., the Code-RPT model and the Math-RPT model) exhibit the largest gains on tasks from their corresponding training domain (Figure D. These results suggest that data following consistent and structured reasoning templates facilitates reliable generalization within the same domain, as downstream tasks can leverage similar inductive patterns.
Unstructured reasoning patterns lack intra-domain consistency. In contrast, models trained on knowledge-intensive domain (unstructured) data demonstrate limited or negative transfer to other unstructured tasks from different domains in our observational study (Figure B). For instance, Finol-88 3B (model (I2)), post-trained on financial data, exhibits notable performance drops when evaluated on all unrelated knowledge-intenss intensive domain tasks. Its pass @ 1 on P PubMedOAA (medical domain) declines from 3.26% to 1. 1.28%, on LegalBench (legal domain) from 6.42% to 4.84%, and TabF act declines from 64.18% to 48.39% (general tabular knowledge). Our interventional results reinforce this observation: the Knowledge-RPP model underperforms the base model on nhowledge-intensive domain tasks, with the degradation in accuracy being statistically significant (Figure 4. This suggests that, unlike structured domains, unstructured reasoning tasks are highly diverse and domain-specifie. They lack shared logical template, making it difficult for RPT to generalize even within what is nominally the same domain.
We provide quantitative evidence for intra-domain diversity of different domains in Appendix B.2
4.4 RPT GENERALIZABILITY REMAINS WEAK ACROSS CONFIGURATION VARIANTS
RPT generalizability is consistent across different algorithms. By comparing the in-domain and out-of-domain performance differences between GRPO and DAPO on the math domain (Table 3 we observe that both algorithms yield similar gain gaps. This suggests that despite their procedural differences, the core optimization behavior of RPT dominates, leading both algorithms to learn essentially the same domain-specine reasoning patterns.
RPT generalizability is consistent across different base models. By comparing the in-domain and out-of-domain performance differences between base models of DeepSeek-R1-DDe --ii---I--iiiil-ie-----. 1,5B and Llama-3.2-3BB 3--tst-ttttt on the math domain (Table 3) we observe that both base models yield similar gain gaps. This suggests that the lack of generalizability in RPT is inherent to the RPT process itself, rather than a consequence of the base model architecture or pretraining data.
RPT generalizability decreases as training steps increase. between in in-domain and out-of-domain gains increases as the number of training steps increases and eventually stabilizes. This indicates that model generalizability gradually declines as RPT progresses. The reason is that increased exposure to the domain-specift data leads the model to overfit, improving in-domaín performance while offering diminishing gains out-of-domain The eventual convergence from a full epoch onwards occurs because the model is repeatedly trained on the same finite collection of data, limiting further changes in the gap betwen in-domain and out-of-domart gains. We conduct more fine- -raiind evaluations and analyses in Appendix H
we show in Figure 5 the gap

---

Published as a conference paper at ICLR 2026
6.02
6:19
1.4
1.37
5.45
1.38
4.91
4.72
20
1.31
1.30
122
1.19
4
A 1000
(/10)
(001gg000)
4000
(0oo
-1
-0.92
0.97
0.96
0.95
9.94
0.5
1.5 # Epochs
2.0
0.5
2.0
Epochs
(a) Pass improvement (D) (%)
(b) Odds ratio () across domains.
Figure 5: In-domain and out-of-domain improvements during RPT training on the math domain, The
gap between in-domain and out-of-doman improvements grows as training progresses.
Larger model sizes do not lead to better generalizability. By comparing performance differences across model-size variants within the same model family, we find that the average in-domain gain increases by 16.5% more than the off-of-domain gain as model size grows. This pattern is consistent with the intuition that larger models are more prone to overfitting the domains which they were RPT-trained, thereby amplifying in-domain improvements without corresponding generalization benefits. We present detailed results and analysis in Appendix
5 RELATED WORK
RPT algorithms and frameworks. RPT, especially RLVR, has received significant attention in building powerful reasoning models, following the release of OpenAl ol (OpenAl al.) 2024) and DeepSeek-R1 (DeepSeek-Al et al., 2025). RPT models have proven effective for a broad range of tasks with well-defined correctness: ranging from structured tasks such as math (Xiong al. 2025; Cui et al. 2025) and coding (Wei et al.) 2025) Shojace et al. 2023 Le et al., 2022] Shen & Zhang 2024), unstructured tasks like search engine use (Jin al. 2025) and open-ended question answering (Su et al., 2025a). However, all these models are trained and evaluated on tasks within single domain or type. Even works on general knowledge (Su et al. 2025a) remain confined to open-ended question answering tasks, without testing transfer across fundamentally different task types. contrast, our work directly addresses this cross-domain generalization gap.
Limitations of RPT. Despite the recent successes of RPT in improving language model reasoning capabilities, the limitations of RPT in general have been widely studied. At the training phase, SFT with LLM reasoning traces, without RI., has been shown to be effective enough (Team) 2025b Yue al. 2025 At the inference phase, the quality of reasoning models depends crucially their ability to scale under test-time compute constraints (Muennighoff et al. 2025 Zhao et al. 2025b). Moreover, the effectiveness of LLMs' lengthy "thinking" processes has also been challenged (Ma al. 2025
Ye et al. 2025). Building on these observations, our work aims to examine the limitations of RPT at a finer granularity, specifically its data generalizability across the training and inference phases.
CONCLUSION
In this paper, we identify important limitations in the generalizability of RPT across domains. Through both observational and interventional studies, consistently find that while RPT produces substantial improvements within training domains, its generalization to unseen domains is limited. In particular, while there is evidence of cross-domain transfer between structured domains like math and code, there is little evidence of transfer to unstructured domains. Our work emphasizes the need for nuanced understanding of cross-domain knowledge transfer in L.MMs.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Alan Agresti. Categorical data analysis. John Wiley & Sons, 2013.
Arcee AL. Arcee maestro-7b-preview
Arcee- Batteee-eettto----PP---
2024. Accessed: 2025-05-12.
Aider-AI. Polyglot benchmark, 2025.
URL
ittps://itthub.com/AAeeer
po 1 yg ot beeccmarrkttabbrreadee----
Anthropic. Claude 3.7 sonnet.
httee://ww..athhhggg...ooo///ooe February
2025.
Anthropic. Claude 4.5 sonnet. httppp//www...tttoooooocco/////a 2025.
Accessed: 2025-11-16.
Anthropic. Introducing claude opus 4.5.
http://www.anttxoopc..cc/m////
claude-opus-44- 4-5 November 2025.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021. BtRLhttps:/ http://axxvvv... org/ harg//bbs. abs/ 2 2108.000000 07 07732
Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran EI- El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp. Ehud Karpas, Ran Zilberstein, Jiaqi Zeng, Soumye Singhal, Alexander Bukharin, Yian Zhang, Tugrul Konuk, Gerald Shen, Ameya Sunil Mahabaleshwarkar, Bilal Kartal, Yoshi Suhara, Olivier Delalleau, Zijia Chen, Zhilin Wang. David Mosallanezhad, Adi Renduchintala, Haifeng Qian, Dima Rekesh, Fei Jia, Somshubra Majumdar, Vahid Noroozi, Wasi Uddin Ahmad, Sean Narenthiran, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Siddhartha Jain, Igor Gitman, Ivan Moshkov, Wei Du, Shubham Toshniwal, George Armstrong, Branislav Kisacanin, Matvei Novikov, Daria Gitman, Evelina Bakhturina, Jane Polak Scowcroft, John Kamalu, Dan Su, Kezhi Kong, Markus Kliegl, Rabech Karimi, Ying Lin, Sanjeev Satheesh, Jupinder Parmar, Pritam Gundecha, Brandon Norick, Joseph Jennings, Shrimai Prabhumoye, Syeda Nahida Akter, Mostofa Patwary, Abhinav Khattar, Deepak Narayanan, Roger Waleffe, Jimmy Zhang, Bor- Yoing Su, Guyue Huang, Terry Kong, Parth Chadha, Sahil Jain, Christine Harvey, Elad Segal, Jining Huang, Sergey Kashirsky, Robert McQueen, Izzy Putterman, George Lam, Arun Venkatesan, Sherry Wu, Vinh Nguyen, Manoj Kilaru, Andrew Wang, Anna Wamo,, Abhilash Somasamudramath, Sandip Bhaskar, Maka Dong, Nave Assaf, Shahar Mor, Omer Ullman Argov, Scot Junkin, Oleksandr Romanenko. Pedro Larroy, Monika Katariya, Marco Rovinelli, Viji Balas, Nicholas Edelman, Anahita Bhiwandiwalla. Muthu Subramaniam. Smita Ithape, Karthik Ramamoorthy, Yuting Wu, Suguna Varshini Velury, Omri Almog, Joyjit Daw, Denys Fridman, Erick Galinkin, Michael Evans, Katherine Luna, Leon Derczynski, Nikki Pope. Eileen Long, Seth Schneider, Guillermo Siman, Tomasz Grzegorzek, Pablo Ribalta, Monika Katariya, Joey Conway, Trisha Saar, Ann Guan, Krzysztof Pawelec, Shyamala Prayaga, Oleksii Kuchaiev, Boris Ginsburg, Oluwatobi Olabiyi, Kari Briski, Jonathan Cohen, Bryan Catanzaro, Jonah Alben, Yonatan Geifman, Eric Chung, and Chris Alexiuk. Llama-nemotron, Efficient reasoning models, 2025. wtwhhtts:// / /ttp/v.ivvvv/./////v//e..////.vv wwg. /tw.b///2555009999
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss. William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. 2021.
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li. Xiyou Zhou, and William Yang Wang. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv: / 1909.0999000 02 164, 2019.
11

---

Published as a conference paper at ICLR 2026
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXhv:2110.14168, 2021.
Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, and Ning Ding. Process reinforcement through implicit rewards, 2025. URL abs/2502. 01456
Quy-Anh Dang and Chris Ngo. Reinforcement learning for reasoning in small Ilms: What works and what doesn't, 2025. StRLhttps::///
.rg.
www.orrg/aaa... abs////////////// ablaaab..50.0.0 16219
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li. H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingchang Chen. Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni. Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang. Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye. Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu. Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong. Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma. Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang. and Zhen Zhang. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
URL htt.hhtt////aiiiii orw. hbtg//bb//2000 12948]
Google DeepMind. Gemini 2.5 pro.
es/
gemmnn//ppi / 2025a.
Google DeepMind. Gemini pro. httppsss/ddeeennn.....go//////// 2025b.
IBM Granite Embedding Team. Granite embedding models, 2024. URL URL.attp::/github.com
1bm- wwwagggegaaanntteeeeeeeeeeeeeeee hww.aaaanttteeeeeeedeneniiinnnno
Maxime Griot, Coralie Hemptinne, Jean Vanderdonckt, and Demet Yuksel. Large Language Models lack essential metacognition for reliable medical reasoning. Nature Communications, 16(1):642, January 2025. ISSN 2041-1723. doi: 10.
URL https:://oi.... org/
00.1038//441457024--111
Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas- Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore. Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai
12

---

Published as a conference paper at ICLR 2026
Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael Livermore, Nikon Rasumov- Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Sharad Goel, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun lyer, and Zehua Li. Legalbench: collaboratively built benchmark for measuring legal reasoning in large language models, 2023.
Zhiwei He, Tian Liang. Jiahao Xu, Qiuzhi Liu, Xingyu Chen. Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang. al. Deepmath- A large-scale, challenging, de- contaminated, and verifiable mathematical dataset for advancing reasoning. arXiv preprint wxXiv:2544.11556 2025.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with apps. NeurIPS, 2021.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12188 2024.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Leaama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024. URL https::tpp/aaiiiis http:///aaaaaaaaooo///////00//
07974
Harold Jeffreys. Theory of Probability. Oxford University Press, Oxford, UK. edition, 1961.
Ming-Hui Jia. Aime 2024, 2024.
URL http:::/huuggnngfccecco///
Maxwe 1 1-Jia/AAAAA AAMME 2024
Jinhao Jiang, Zhipeng Chen, Yingqian Min, Jie Chen, Xiaoxue Cheng, Jiapeng Wang, Yiru Tang, Haoxiang Sun, Jia Deng, Wayne Xin Zhao, Zheng Liu, Dong Yan, Jian Xie, Zhongyuan Wang, and Ji-Rong Wen. Enhancing Ilm reasoning with swwrdd-guieed tree search. arXiv preprint 2rXiv:2411.11694, 2024.
Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024. URL
https: www.:vrrgaaaao. abs/ /obos/23100 2310006770
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang. Hamed Zamani, and Jiawei Han. Search-rl: Training 1lms reason and leverage search engines with reinforcement learning, 2025. URL http::::::// //wrxvaaaivvvv//////ii org/ abs 250..
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint 1rXiv:1909.06146, 2019.
Hynek Kydlíček and Greg Gandenberger. Math-verify: robust mathematical expression evaluation system, 2025.
Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven C. H. Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning, 2022. URL
https: org. wttpr///bb/222220022
Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi. Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://hugginng http://ww..ooooooottat./t
2024.
Rongao Li, Jie Fu, Ben Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. Taco: Topics in algorithmic code generation dataset. arXiv preprint arXiv:2312. 14852, 2023.
13

---

Published as a conference paper at ICLR 2026
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2023.
Zichen Liu, Changyu Chen, Wenjun Li. Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding -l-zroo-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025.
Locutusque. Thespis-llama-31193
Thespis aallammm333 1-BB 2024.
Michael Luo, Sijun Tan, Roy Huang, Xiaoxiang Shi, Rachel Xin, Colin Cai, Ameen Patel, Alpay Ariyak, Qingyang Wu, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder. fully open-source 14b coder at 03-mini level, 2025a. Notion Blog.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol ol-preview with a 1.5b model by scaling rl, 2025b. Notion Blog.
Chengqi Lyu, Songyang Gao, Yuzhe Gu, Wenwei Zhang, Jianfei Gao, Kuikun Liu, Ziyi Wang. Shuaibin Li, Qian Zhao, Haian Huang, et al. Exploring the limit of outcome reward for learning mathematical reasoning. arXiv preprint arXiv:2502. 06781, 2025.
Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking, 2025. URL htw.t:tt//s/t... //aro::////iiiii http://axiiv..gg//////////0 09858
Codeforces python submissions, 2024. URL https://huggiigggfcees co/ ort.eees//aaaassccccoo/ooooeeeee
MatrixStudio.
Meta AI Llama 3.2: Revolutionizing edge
and vision with open,
customizable models, 2024a.
URL https://al http://aaaaaaaaaaaaa/eeam mwt.. com.tooo///oo..ooo /log/
llama 3- 2-coneect 2024-vision-edae mode-n--eeeee-eeeeeeeeeoeoooeeee mobile mobille-eeieeesee devices/ Accessed: 2025-11
15.
Meta AI. Meta llama 3.1 instruct.
LLlama
1 8B Instruct] 2024b. Release date: July 23, 2024.
Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems. arXiv preprint arXiv:2412.0944 2024.
mlabonne. hitallllmmn-11------sseeeeaalaie https:://uggiiiii
Meta Llama 1- Instruct hat.aaa..---ssstttutttaaaaaeeeee
2024.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL /tt.://aaee///aa...oo////////...
Skywork ol Team. Skywork-ol open series htgp:///w..ooooooaoooo////////// ber 2024. /ttt////ttpp////ggggggee////////
Novem-
OpenAl. Gpt-4o. http:///sh/////paaaaam//////////
May 2024.
OpenAl. o3 and o4-mini system hra.amaaaaaaaaaaoa....aa...00na0 pdf
OpenAl. Gpt-5.2 pro model documentation.
doca/mode ltsooddell/pppp / -.ppttt5..pppp pro 2025.
14

---

Published as a conference paper at ICLR 2026
OpenAI, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applehaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghor- bani, Ben Rossen, Benjamin Sokolowsky. Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman. Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy. Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang. Edmund Wong, Elizabeth Proehl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang. Felipe Petroski Such, Filippo Raso, Florencia Lconi, Foivos Tsimpourlas, Francis Song. Fred von Lohmann. Freddie Sulit, Geoff Salmon, Giambatista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclere, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O'Connell, Ian Osband, Ignasi Clavera Gilaberte, Ilge Akkaya, Ilya Kostrikov, Ilya Sutskever, Irina Kofman, Jakub Pachocki, James Lennon, Jason Wei, Jean Harb, Jerry Twore. Jiacheng Feng, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joaquin Quiñonero Candela, Joe Palermo. Joel Parish, Johannes Heidecke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga. Julie Wang, Kai Chen. Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Leeberrg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang. Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng. Linden Li, Lindsay McCallum, Lindsey Held, Lorenz Kuhn, Lukas Kondraciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tintor, Mason Meyer, Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yatbaz, Melody Y. Guan, Mengyuan Xu, Mengyuan Yan, Mia Glaese, Mianna Chen, Michael Lampe, Michael Malek, Michele Wang. Michelle Fradin, Mike McClay, Mikhail Pavlov, Miles Wang, Mingxuan Wang, Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowd- hury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao. Paul Ashbourne, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes, Raz Gaon. Reah Miyara, Reimar Leike, Renny Hwang, Rhythm Garg, Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer, Samuel Miserendino, Sandhini Agarwal, Santiago Hernandez. Sasha Baker, Scott McKinney, Scottie Yan, Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang, Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji, Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang, Taylor Gordon, Ted Sanders, Tejal Patwardhan, Thibault Sottiaux, Thomas Degry, Thomas Dimson, Tianhao Zheng, Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi. Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong, Vlad Fomenko, Weiyi Zheng, Wenda Zhou, Wes McCabe, Wojciech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha, Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang, Zheng Shao, and Zhuohan Li. Openai ol system card, http:/:///xxwv./g///ea/
2024. htt.:///c...ooaaaa...e..e/errrra AAMMMP///..../ASAA51.. 2025.
OpenCompass Community. opencompaas/aiie2225:: Aime 2025 dataset.
https:
Lingfei Qian, Weipeng Zhou, Yan Wang, Xueqing Peng, Jimin Huang, and Qianqian Xie. Finol: On the transfcrabiliity of reasoning enhanced llms to finance. arXiv preprint arxiv:2502.081127, 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level prof a& benchmark. arXiv preprint 2rXiv:2311. 12022, 2023. URL httphhttt:///attt wwtp:///ggg//bb/bb///bo/se U
12022
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang. YK Li, Y Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Wei Shen and Chuheng Zhang. Policy filtration in rlhf to fine-tune Ilm for code generation, 2024. htll:htts::////t wwt.://wrwabbb....osaa
15

---

Published as a conference paper at ICLR 2026
Quan Shi, Michael Tang, Karthik Narasimhan, and Shunyu Yao. Can language models solve olympiad programming? arXiv preprint 2.KYv:2404.19952, 2024.
Parshin Shojaee, Aneesh Jain, Sindhu Tipirenii, and Chandan K. Reddy. Execution- based code generation using deep reinforcement learning, 2023.
13816
Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. Crossing the reward bridge: Expanding with verifiable rewards across diverse domains, 2025a.
URL httthhtt///axxtttt wrt. abw.ab/3..... 2582.
Yi Su, Dian Yu, Linfeng Song, Juntao Li. Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. Expanding rl with verifiable rewards across diverse domains. arXiv preprint arXiv:2503.23829 2025b.
NovaSky Team. Unlocking the potential of reinforcement learning in improving reasoning models. hutp::/novaaky-aiigihhuul
2025a. Accessed: 2025-02-13.
Open Thoughts Team. Open Thoughts, January 2025b.
Qwen Team. Qwen3 technical report, 2025c. URL httpp://aarxiiroorrraaa///// 09388
RUCAIBox STILL Team. Still-3-1.50 --..piiirrre Enhancing slow thinking abilities of small models
through reinforcement learning. 2025d. URL Tttps:///gttuuu..cm//RCAA.L.LL.
L.MM
Lewis Tunstall and Li Jia. Ame 2023, 2024. URL http:::/nugginnfaccc.oooaaa
AI-MO/a a i va 1
amc.
Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. Swe-rl: Advancing 1lm reasoning via reinforcement learning open software evolution, 2025. URL /tR.sttaa//raai..or////////
xAI. Grok 3 beta the age of reasoning agents. 2025.
February
Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing Ilm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502. 14768, 2025.
Wei Xiong, Hanning Zhang, Chenlu Ye, Lichang Chen, Nan Jiang, and Tong Zhang. Self- rewarding correction for mathematical reasoning, 2025. URL htttp:://arxii arabbaag/oooo./////oo/a////a 25C2.1961.
org/abs/
Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran. Kodcode: A diverse, challenging, and verifiable synthetic dataset for coding. arXiv preprint wxXiv:2503.0295:: 2025.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang. Jianxin Ma. Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang. Ru Peng. Rui Men, Ruize Gao. Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint arXiv:2407.. 10671, 2024.
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less more for reasoning, 2025, URL ht //apiiv//a.iiiaa//a org.ivv..or..ooooooooooooo...... abs 25022 03387
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
16

---

Published as a conference paper at ICLR 2026
Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels. arXiv preprint TrXiv:2412.0198/1 2024.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang.
Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model?, 2025. URL
Zhihan Zhang, Yixin Cao, and Lizi Liao. Finbench: Benchmarking Ilms in complex financial problem solving and reasoning.
Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Yang Yue, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data, 2025a.
03335.
Eric Zhao, Pranjal Awasthi, and Sreenivas Gollapudi. Sample, scrutinize and scale: Effective inference-time search by scaling verification, 2025b. URL Uttttts:://axxi org/ /asr//b//2500 01839
Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen Gong, Thong Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex Gu, Zhoujun Cheng. Jiawei Liu, Qian Liu, Zijian Wang, Binyuan Hui, Niklas Muennighoff, David Lo, Daniel Fried, Xiaoning Du, Harm de Vries, and Leandro Von Werra. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions, 2025. URL Lttppppp../aaxxx....o///////0/0.
Zyphra. Zyphra -1.-b. http:://aaggginooooo..o////////a 2024. Accessed: 2025-05-12.
17

---

Published as a conference paper at ICLR 2026
A USE OF LARGE LANGUAGE MODELS (LLMS)
No LLMs were used in the ideation, writing, or preparation of this paper. All content was conceived, drafted, and revised solely by the authors.
B REASONING TEMPLATE ANALYSIS
To better interpret the generalizability across domains, we analyze the reasoning templates used in different types of tasks. Specifically, we randomly sampled min (dataset size, 100) tasks from each evaluation benchmark, resulting in 1,470 task instances total. We then used Claude Sonnet 4.5 (Anthropic] 2025), the state-of-the-art periphery reasoning model with exposed reasoning traces, to complete the selected tasks. Next, we collected the generated reasoning traces and used GPT- 4o (OpenAT 2024) to tag each step of the reasoning traces using the following taxonomy:
READ_RESTAN Restating or understanding the question.
SETUP: Defining variables, listing assumptions, establishing context, initializing struc- tures.
PLAN: High-level strategy, decomposition, or outlining an algorithm.
EXECUTE STEP: Any intermediate logical, mathematical, algorithmic, computational, or transformational operation.
SESCUTE_STEP.
CONTROL FLOW: Branching, case analysis, loops, recursion, considering alternative scenarios.
VERIFY:
Testing, checking correctness, or performing sanity checks.
FINAL_ ANSWER: Final conclusion or solution.
OTHER: Anything not fitting the categories above.
Finally, we quantitatively analyzed the divergence of the reasoning templates in the tagged traces within and across domains. We used the Jeffreys divergence (Jeffreys 1961 measure the symmetric divergence between step-type distributions.
We present the detailed distribution of each domain in Figure 6 We observe that the annotations are of high quality: almost no steps are labeled as OTHER, indicating that our tagset provides comprehensive coverage of the reasoning process. Building on this, find that EXECUTE_ STEP the most dominant category across all three domains. PLAN particularly important math and code tasks, reflecting their structured, multi-step problem-solving nature, while SETUP plays a comparatively larger role in cowweede-intensiig tasks, where establishing context or recalling background information crucial.
_
We now use the distribution vectors to quantify our hypothesis in Section
B. MATH AND CODE DOMAINS SHARE SIMILAR REASONING TEMPLATES
We computed the Jeffreys divergence (Jeffreys) 196T) between two domain- reasoning-step distributions, which is obtained by aggregating the tagged traces of cach domain and normalizing single-domain traces into probability distributions. Using these aggregated distributions, the Jeffreys divergences 0.18 (between math and code), 0.29 (between math and nowlledgeintensiive 0.69 (between code and -eeeedgg--neensive........ This indicates that math and code share more similar reasoning templates, while knowledge-intensive tasks require substantially different reasoning.
and
B.2 KNOWLEDGE- INTENSIVE DOMAIN HAS LARGER INTRA- DORMANN VARIANCE ($4.3)
We compute the Jeffreys divergence between every pair of datasets within each domain, using the normalized reasoning step distributions for each dataset. Averaging over all pairwise comparisons, the Jeffreys divergences 0.15 (within the math domain), 0.14 (within the code domain), and 0.19 (within the knowledge intensive domain). This indicates that -edee--ntenssive
intensivee tasks exhibit a more diverse set of reasoning templates compared to math and code tasks.
18

---

Published as a conference paper at ICLR 2026
Math
Code
0.35
0.34
Knowledge
0.31
0.30
0.25
0.16
0.14
0.34
0.13
0.10
0.08
0.05
0.05
0.04
0.07
0.06
0.06
0.07
0.03
0.01
0.00
READ RESTATE
SETUP
PLAN
EXECUTE_ STEP
CONTROL FLOW
VERIFY
FiNAL ANSWER
OTHER
Figure 6: Distribution of reasoning-step tags across domains.
Table 4: Larger model size does not improve generalizability, as shown by accuracy gains A (%) and odds ratios 9 on in-domain (ID) tasks compared to out-of-domain (OOD) tasks across different size variants. For difference- between the larger and smaller variant (Large Small); for ratio-based metrics, the model size effect computed as the average ratio (Large Small).
metries, the model size effect computed as the average difference
Model
A(D))
A(OOD) t AUD) A (ODD)
(UDD) t
(LOOD)
(LID) (0(OOD)
DeepCoder- 1.5B
4.61
4.01
0.60
1,4496"
1.1840"
1.2243
DeepCoder-14B
-1.87
-5.68
3.81
0.8715'
0.7122
1.2238
OREAL-7B
-0.41
8.44
-8.85
0.9714
1.4724
0,6597
OREAL-32B
0.61
-0.04
0.65
1.0571
0.9982
1.0590
Finol-8B
-3.84
-27889
24.05
0.8319"
0.3034"
2.7420
Fin-o1-14B
-3.60
-5.43
1.83
0.8481*
0.7047
1.2034
AZR-Coder-3B
-6.27
2.55
-8.82
0.5235
1.1517"
2.7420
7ER-Coder-7B
30.12
-38880
68.92
22.4671*
0.2452
91.6456
A7R- PZR-Coder-11e 14B
25.01
24.34
0.67
15.5878"
8.5023"
8,5023
Model Size Effects
9.56
7.98
1.58
12.683
7.645
6.610
C RPT GENERALIZABILITY ON MODELS ACROSS DIFFERENT SIZES
By examining the performance differences across model variants in Table we observe no clear trend indicating improved generalizability with larger models. Several models exhibit reduced generalizability as size increases. We believe this is because larger models more effectively overfit to the domains on which they were RPT-trained. The primary exception the Fino family, whose improved generalizability is explained by change in base model rather than model size itself (Finol-8B is based on Llama, whereas Fin-ol-14p uses Qwen3-14B).
D REWARD SIGNALS DURING INTERVENTIONAL STUDY STUDY
D.1 REWARD DEFINITIONS
Following prior work that has demonstrated promising performance (Luo et al. 2025a Yu et al.
2025). we use specifc binary reward functions:
Math: if model's answer is mathematically equivalent the ground truth; otherwise 0.
19

---

Published as a conference paper at ICLR 2026
6.43
6:34
0.30
Taklin MAware
6:33
0.30
6:45
0.22
6:88
6.03
Toinngg Strpe
Toinng Stops
Thaiiing Strips
(a) Math
(b) Code
(c) Knowledge-intensive
Figure Training rewards of our RPT training runs for math, code, and knowledge wowleeggeiitensive domains. Over the 625-step training process, the 50-step moving average training rewards of math, code, and knowledge-intenast domains increase by 7%, 21.7%, and 71.0%, respectively.
Table 5: Mixed-domain RPT performance compared the best single-domain RPT model. The effects column reports differences for (DD) (Mixed Min) and ratios for ((D) (Mixed Min).
Mixed-RPT
Min(RPT)
Effects
(Knowkdge)
f
-2.17
-3.98
1.81
(Code)
1.91
-0.02
1.93
(Math)
-3.62
-0.15
-3.47
(nowledge)
0.91
0.88*
1.03
(Cooe)
1.18
1.00
1.18
(Maah))
0.83*
0.98
0.85
Code: 1 if model's answer passes all unit tests; otherwise 0.
Knowledge: if model's answer matches the ground truth string; otherwise 0.
For extraction, we symbolic and string-based equivalence checking for math via the math-verify package (Kydličck & Gandenberger 2025), unit tests for code, and string-based checking for general knowledge. To quantitatively assess reward quality, we sampled 100 training examples and manually verified the correctness of their reward signals. The correctness rates for math, code, and general knowledge are 94%, 90%, and 97%, respectively, indicating consistently high reward quality across domains.
D.2 OPTIMIZATION DYNAMICS
In this section, we show the optimization dynamics our RPT training runs. We present the training rewards for each domain in Figure 7 For each domain, we show the raw training rewards and the moving average rewards over 50-step windows.
As shown in the figures, we find that the training rewards of all domains exhibit noticeable increase over the course of RPT training. Quantitative, the 50-step average training rewards math, code, and knowledge-intensie domains increase by 9.7%, 21.7%, and 71.0%. This shows that the model learned domain -specifcc reasoning techniques single- RPTs, which in turn improves the task-specific performance.
E MIXED DOMAIN TRAINING
We train an additional model using mixed-domain dataset, created by randomly sampling 13,333 data points from each the single-domain datasets. As shown Table [ the mixed-domain RPT model achieves overall 2.7 percentage-point accuracy improvement compared to individual single-domain RPT models. models.Importanty. Importantly, this mixed-domain result should be interpreted as lower bound: because simply drew small, uniformly sampled subset from each domain without any optimization, a more principled mixed-domen curation strategy could potentially yield even stronger performance.
20

---

Published as a conference paper at ICLR 2026
F
PER MODEL PER TASK RESULTS
F.1 OBSERVATIONAL. STUDY RESULTS
Table 6: Observational study results. The model IDs corresponds to Table I
Model
pubmodqa modga aime2024 gsmslk math5000 ame25 fab_ legilbench finhen livecodetench codefoocee polyplot hananeval higcoddbesch mbpp usaco
1-RPT
0.640
0.242
0.331
0.823
0.892
0.697
0.397
0.547
0.230
0.009
0.713
0.126
0.511 0.065
Buse
0.616
0.261
0.279
0.747
0.842
0.683
0.568
0.386
0.520
0.175
0.024
0.000
0.652
0.112
0.474 0.052
2-RPT
0.632
0.242
0.375
0.848
0.906
0.711
0.714
0.617
0.580
0.235
0.013
0.013
0.701
0.130
0.584 0.055
2 Base
0.616
0.261
0.279
0.747
0.842
0.083
0.668
0.586
0.520
0.175
0.024
0.000
0.652
0,112
0.474 0.082
SRPP
0.214
0.005
0,005
0.885
0.798
0.497
0.071
0.330
0.176
0.150
0.027
0.049
0.780
0.425
0.647 0u029
3-Bose
0.326
0.266
0.029
0,766
0,464
0.323
0.642
0.611
0.316
0.110
0.009
0.009
0.610
0.397
0.675 0.016
4-PPT
0.710
0.355
0.177
0.917
0.774
0.648
0.688
0.537
0.524
0.188
0.021
0.009
0.541
0.172
0.621 0.033
4-Rose
0.714
0.401
0.062
0.812
0.080
0.405
0.620
0.110
0.004
0.543
0.079
0.589
0.013
S-RPT
0.508
0.250
0.010
0.424
0.370
0.125
0.589
0.200
0.360
0,052
0.003
0.009
0.000
0.073
0,560 0.000
Base
0.474
0.211
0.006
0.434
0.412
0.110
0.460
0.159
0.388
0.043
0.003
0.013
0.268
0.179
0.673 0.003
RPP
0.592
0.395
0.016
0.801
0.662
0.369
0.76]
0.175
0.501
0.128
0.013
0.378
0.382
0,768 0.029
Buse
0,622
0.233
0.008
0.114
0.496
0.341
0.228
0.617
0.580
0.013
0.001
0.004
0.104
0.068
0.005 0.007
7-RPT
0.668
0.365
0.317
0.804
0.864
0.683
0,700
0.000
0.366
0,200
0.036
0.004
0.640
0.102
0.552 0.055
7 Base
0.616
0.261
0.279
0.747
0.842
0.683
0.668
0.5 0.586
0.520
0.175
0.024
0.000
0.652
0.112
0.474 0.052
SRPP
0.520
0.316
0.562
0.718
0.952
0.745
0.460
0.624
0.003
0.385
0.170
0.000
0.848
0.357
0.597 0.288
8-Rose
0.326
0.366
0.029
0.766
0.464
0.323
0.642
0.611
0.316
0.110
0.009
0.009
0.640
0.397
0.675 0.016
9-RPT
0.724
01501
0.015
0.636
0.448
0.151
0.615
0.230
0.587
0,050
0.004
0.004
0.189
0.109
0.129 0.007
9-Base
0.732
0.496
0.008
0.695
0.224
0.120
0.616
0.008
0.000
0.573
0.261
0.271 0.000
10 RPT
0.620
0.245
0.294
0.809
0.862
0.727
0,650
0.558
0.541
0.175
0.021
0.000
0.622
0.082
0,518 0.039
10 1--Base
0,616
0.261
0.279
0.747
0.842
0.683
0.586
0.520
0.175
0.000
0,652
0.112
0.474 0.052
11-RPT
0.740
0.304
0.367
0:911
0.924
0.825
0.791
0 0.000
0.603
0,311
0.063
0.004
0.817
0.351
0,810
11-Base
0.740
0.324
0.581
0.914
0.940
0.008
0.808
0.748
0.738
0,328
0.094
0.027
0.817
0.320
0.826 0.
12-RPT
0.128
0.304
0.029
0.753
0.414
0.227
0,484
0.287
0.277
0.018
0.002
0.049
0.543
0.360
0.630 0.010
12. Rase
0.326
0.766
0.029
0.766
0. 164
0.323
0.642
0.611
0.316
0.110
0.009
0.009
0.640
0.397
0.675 0.016
12: RPT
0.368
0.110
0.263
0.898
0.802
0.728
0,276
0.566
0,361
0.068
0.010
0.018
0.494
0.164
0.809 0.016
13-Base
0.144
0.088
0.273
0.911
0.816
0.700
0.086
0.561
0.138
0,050
0.006
0.018
0.354
0.138
0.204 0.013
1I-PPT
0.614
0.231
0.263
0.754
0.846
0.630
0.626
0.584
0.529
0.147
0.025
0.009
0.622
0.099
0.468 0.036
14 Base
0.616
0.261
0.279
0.747
0.842
0.683
0.668
0.586
0.520
0.175
0.024
0.000
0.652
0.112
0.474 0.052
15-RPT
0.779
0.791
0.606
0.947
0.794
0.927
0.887
0.821
0.500
0.357
0,110
0.000
0.896
0.259
0.889 0.238
15-Rase
0.775
0.652
0:951
0.952
0.893
0.259
0.896
16-RPT
0.552
0.183
0.444
0.941
0.898
0.839
0.234
0.600
0.337
0.165
0,089
0.000
0.104
0.000
0.724 0.085
16-Base
0.385
0.156
0.421
0.948
0.886
0.823
0.332
0.177
0.039
0.000
0.652
0.154
0.718 0.094
17-RPT
0.720
0.683
0.221
0.923
0.750
0.664
0.813
0.659
0.282
0.065
0.004
0.598
0.246
0.807 0.147
17-Base
0.763
0.825
0.675
0.951
0.504
0.945
0:904
0.820
0.695
0.465
0.156
0.000
0.945
0.230
0.895 0.423
18-RPT
0.632
0.401
0.023
0.083
0.180
0.228
0.647
0.107
0.606
0.113
0.007
0.009
0.307
0.184
0.817 0.036
18 Base
0.076
0.052
0.002
0.245
0.056
0.078
0.043
0.010
0.060
0.000
0.000
0.018
0.152
0.050
0,050 0.003
F.2 INTERVENTIONAL STUDY RESULTS
Table 7: Full interventional evaluation results across models and benchmarks.
Medel
pobmerkgs masige aimc2004
parkk mati:500
amc33 tat: fact legalbennh fabhen licandshanch endefoeccc polyplet humancaal ligensthench mhyp usaen
Math RPT
a.636
0,233
0.319
0,389
0.811
0,685
0.678
0.585
0.447
0,203
0.004
0.013
0.689
0.109
0.514 0,055
Code RPT
0.622
0.222
0.296
0,000
0.847
0.665
0.664
0.591
0,472
0.158
0.024
0.008
0.530
0.253
0.500 0,000
Knowletge-RPP
0.66
0,221
0,325
0,557
0.852
0.685
0,656
0.584
0,414
0,183
0.025
0.008
0.591
0,100
0.493 0,006
Malh DAP-
0.642
0.282
0.333
0.318
0.724
0,770
0.664
0.555
0.475
0.172
0.000
0.004
0.555
0.070
0.496 0.042
Mall-PP--- Lama
0558
0.087
0.588
0:373
0.662
0.855
0.400
0.085
0.008
0.000
0.500
0.145
0.965 0.005
Math OP epoch
0.600
0.305
0,333
0.304
0.730
0.759
0.661
0.585
0.500
0.200
0.025
0,000
0.585
0.66
0.485 0:009
Math RPT
0.632
0.308
0.321
0,832
0.726
0,775
0.662
0.584
0.49]
0.025
0.004
0.600
0063
0.886 0,042
Math NPT epochs
0.655
0,315
0.340
0.347
0/226
0.241
0.062
01586
0,477
0.024
0.04
0.699
0.075
0.16 0,046
Math-RLPT-3
0.646
0.284
0.331
0.836
0.728
0,762
0.638
0.535
0.469
0.200
0.325
0.000
0.604
0.069
0.457 0.002
Mixzd-RPT
0.665
0.385
0.127
0.727
0.660
0.799
0.679
0.540
0.448
0.207
0.040
0.004
0.64
0.064
0.564 0.052
Bise: Owen
0.616
0.261
0.279
0.347
0.842
0.683
0.668
0.586
0.520
0.024
0.000
0.652
0.112
0.474 0.052
Buse: Liana
0.467
0.044
0.444
0.239
0.587
0.005
0.494
0.153
0.561 0.007
21

---

Published as a conference paper at ICLR 2026
G RELATIVE IMPROVEMENT
To address the importance of relative performance gains, we complement our absolute aggregated improvement measure with secondary metric that captures proportional improvements while remaining well-defined even when the base model attains zero accuracy.
Formally, report the relative aggregated improvement.
D0 ELED NiRe Pa.j.
(1)
ExED Ne
where the per- berchenchmark relative gain is defined as
Ai, - Astt
Aj,r > 0,
Phgt
Ajt
(2)
Ai,
Aj,ts
A =
We now include the results we reported with relative improvements. We can see that our conclusion further strengthened with this secondary metric. We illustrate findings follows.
The gap between in-domain and out-of-domain tasks is more evident under relative improve- ments. Table Taaleaecompares the in-domain and out-of-domain Pass @ relative improvements model pairs in the observational study. The relative gap is 256.71 1,, indicating that accuracy on in-domain tasks increases more than 3.5x x on out-of-doman tasks. This contrast is substantially sharper than what is shown by the primary metrics in Table 2 where the absolute accuracy improvement is 6.07%.
The same pattern appears consistently across all settings. Relative improvements highlight larger separations between domains in the observational study (Figure (8) compared the accuracy-basee results (Figure3)) The effect also holds for the interventional study when comparing Figures and
10 with their absolute counterparts in Figures and
Overall, relative improvement serves complementary metric that amplifies domain-win gaps of RPT model performance, and these clearer separations further strengthen our conclusion that RPT lacks generalizability across domains.
Table 8: Relative Pass @ improvements (%) for in-domain and out-of-domain tasks across model pairs in the observational study.
121
17)
(4)
(8)
15.
176)
1171
(12)
173)
(14)
(25)
(761
(37)
(/A)
Any.
7.00
.4.57
525.15
10.32
-31.83
0.65
90.61
3009
20000)
602
2.88
$50
-10.08
42.86
-2.23
38,01
64.95
-800
0.30
133
(2)
150
681
(3)
18)
171
180
69
(201
03
(22)
1131
(44)
(251
0351
(27
12:01
Figure 8: Relative Pass @ improvements (%) across domains for model pairs in the observational study.
The major RPT configuration for interventional analysis achieves the strongest generalizability As shown in Table our main configuration (Qwen GRPO) exhibits a substantially smaller relative accuracy gap than the other two variants. This contrast becomes especially clear under the relative- improvement metrie: the Llama-based configuration yields gap of 28. 18.38% whereas the Qwen-based configuration yields only 8.75%. By comparison, the absolute accuracy gaps reported in Table differ only slightly (4.94% 5.23%). These results demonstrate that adjusting RPT implementation variants does not improve generalizability. They also validate the design and experimental setup of interventional study.
22

---

Published as a conference paper at ICLR 2026
Domain
Out of Domaln
Knowledge
20
Model
2.18
24.70
Code
14 66
24.70
-5.02
10
Model
Khowledge
5.97
Math
5.38
-2.31
3.53
30
15
20
25
Math-RPT
Code-RPT
Knowledge-RPI
Passig1 relud ve imprevement 1941
Figure 10: Relative @ass improvements (%) across domains for model pairs in the interven- tional study.
Figure 9: Relative Pass@1 improvements (%) for in-domain and out-of-domain tasks across model pairs in the interventional study.
Table 9: RPT configuration variants consistently fail to im- prove generalizability, as shown by relative Pass@) improve- ments A (%) on in-domain (ID) tasks versus out-of-domain (OOD) tasks across different base models and RPT algo- rithms.
7,88
9:27
6.72
4
A900)
1.36
10
Base Model RPT Algoritha
((O0))
Sports
DeepSeck-R Distill D-i-iiill--uen-tttt Qwen- 1,58 GRPO
5.58
-3.16
8.75
Llama-3.2-38 2aaim-3.---B-nsttuuee GRPO
28.38
Figure 11: Relative Pass @ improve-
DeepSeck RI RI-Distil-QQwen Selll--eee--..66 + DAPO
7881
-2.43
10.24
ments (%) across checkpoints.
The effect of training epochs becomes more evident, and the convergence frend is clearer with relative improvements. As shown in Figure 11 the relative performance first increases and then stabilizes as the number RPT epochs grows. Compared Figure the relative-improvemenn view makes both the growth phase and the convergence pattern more pronounced. This further highlights how the model increasingly overfits the training domain and lacks generalizability.
the two primary metrics reported in
H CHECKPOINT EVALUATIONS AT 100-STEP INTERVALS
1.4
1.3
1.2
(OOD)
gooco
(000
(000)
1.1
Booyyrooos
1.0
250
500
750
1000
1250
250
500
750
1000
1250
# Training Steps
Training Steps
(a) @ improvement A (2) (%)
(b) Odds ratio @(P) across domains.
Figure 12: In-domain and out-of-domain improvements 100-step intervals during RPT training the math domain. full epoch corresponds to 600 training steps. The gap between in-domain and out-of-domain improvements grows training progresses. We report the specific values in Table TO
23

---

Published as a conference paper at ICLR 2026
We perform finer-graine's analysis of model behavior by evaluating intermediate checkpoints 100-step intervals throughout RPT training. We display the results in Figure 12 and Table 10 These results further strengthen conclusion in Section 4.4 that as training progresses, the generalizability of RPT decreases. Specifically, we observe that:
in-domain gains remain consistently positive, and out-of-domain gains remain consistently negative;
2. in-domain gains increase, whereas out-of-domain gains decrease;
3. the gap between in-domain and out-of-domain gains widens as training advances; and
4. the trends eventually stabilize, partieularly after the end of the first epoch, when the model has been exposed to the full training dataset.
Table 10: Math-RPT checkpoint performance at 100-step intervals, reporting accuracy gains A (%) and odds ratios 0 for in-domain (ID) and out-of-domain (OOD) tasks. full epoch corresponds to 600 training steps.
Model
AUD) f
A (000) f
AUD)
A (OOD)
((ZD) f
((OOD) f
((D)) (g(OOD)
Math-RPT-100 100-steps
1.74
-0.31
2.06
L,0688
0.9869
1.1113
Math-RPP-200-steer
2.07
-0.47
2.54
1.1164
0.9806
1.1386
Math-RPT-300-steps
3.19
-0.63
3.82
1.1870"
0.9738
1.2189
Math-RPTT4400-sepps
4.14
-0.86
5.00
1.2517
0.9645
1.2977
Muth-RPT-5000seeps
3,41
-0.85
4.26
1.2027
0.9649*
1.2464
Muth-RPT-600-stepee
4.53
-0.92
5,45
1.2833
0.9623
1.3336
Math-RPT-770-stee
3.93
-0.95
4,88
1.2373
0.9609
1,2876
Math-RPT-800---
4.15
-1.33
5,48
1.2556
0.9459
1.3275
Math-RP7-9900-stpps
4.91
-1.11
6.02
1.3100
0.9544
1.3725
Math-RPTT-000-stees
5.26
-1.43
6.68
1.3353
0.9419
1.4177
Math-RPPT-1100sseeee
4.43
-1.40
5,82
1.2756*
0.9430
1.3527
4.72
-1.47
6.19
1.2962
0.9401
1.3789
24


Reviews:
[{"review_id": "x7J7rHR1oU", "reviewer": "Reviewer_172Y", "summary": "This paper presents an empirical study that assess to which degree reinforcement learning post-training enables generalizable improvement of reasoning across domains. It is structured into two parts, an observational and an interventional study.", "questions": "The issues to be discussed in my opinion can be derived straightforward from the weaknesses section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The topic addresses a currently open important question for reasoning LLMs.\n\nCertainly a strength of the paper is its systematic and transparent setup (in particular for the selection of tested models) and statistical evaluation.", "weaknesses": "A key weakness of the study is the focus on small models. While I understand the computational limitations. However, it seems reasonable that a certain model complexity might be required to actually generalize across domains. Therefore, it is not clear how the findings actually generalize to larger models that might in any case better suited for complex reasoning tasks.\n\nSimilarly, only one particular Reinforcement Learning process is tested for fine tuning with a single snapshot after one epoch. Here, it would be key to also see the development over multiple snapshots. With the current setup, one could hypothesize that generalization just sets in later. An evaluation would be interesting.\n\nThe paper overall is very sparse with the exact evaluation results for the different tests and models. I would expect that the paper reports on the detailed per task per model accuracies.\n\nThe used evaluation measure is fine. However, I think there is a large difference between an increase of accuracy from 60% to 61% or an improvement from 95% to 96%. In other words, the relative improvement is also important and should be addressed in a second measure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IA2utca9hl", "reviewer": "Reviewer_TWrh", "summary": "The paper examines whether reasoning gains from RPT generalize beyond the training domains. Through both observational and controlled interventional studies across math, code, and knowledge-intensive tasks, the authors find that RPT improvements are domain-specific, effective within similar structured domains, eg. math and code, but failing to transfer to unstructured ones, e.g., legal, medical. The work highlights the limited cross-domain generalizability of current RPT approaches.", "questions": "- In the interventional experiments, were the three single-domain RPT models trained with identical reward functions or domain-specific ones? Clarifying such details could help interpret whether the observed generalization gaps stem from reward differences or reasoning differences.\n- Could the authors comment on whether a mixed-domain RPT training setting, e.g., combining math, code, and knowledge reasoning, might mitigate the observed specialization? This would help verify whether domain isolation itself causes the loss of generalization.\n- Have the authors considered analyzing intermediate checkpoints during RPT training to see if cross-domain performance degrades gradually or abruptly? Such temporal analysis might shed light on when specialization emerges.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper tackles an important and timely question about whether reasoning improvements from reinforcement post-training can truly generalize beyond the training domain.\n- The study design is comprehensive and convincing, combining large-scale observational analysis of public RPT models with controlled interventional experiments under unified settings.\n- The experiments are extensive and well-documented, covering 16 diverse benchmarks across mathematics, code, and knowledge reasoning with appropriate statistical validation.", "weaknesses": "- The experiments are conducted on relatively small models (up to 8B) with limited-scale RPT training, leaving it unclear whether the same generalization patterns would persist under larger LLMs.\n- The paper stops short of analyzing how different aspects of RPT training, such as reward signal quality or optimization dynamics, might contribute to the observed lack of cross-domain transfer, leaving the underlying cause somewhat underexplored.\n- The paper does not include any longitudinal or ablation analysis during training, which could reveal how generalization patterns evolve over time or collapse across domains.\n- The interventional experiments are all based on a single backbone DeepSeek-R1-Distill-Qwen-1.5B), so the conclusions are lacking in generality as the observed trends may depend on that model’s pre-training distribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MCpcLPCUWk", "reviewer": "Reviewer_8H9o", "summary": "This paper conducts an extensive study on publicly released models to understand the cross-domain skill transfer during reinforcement learning. They explore mathematical, code, and knowledge-intensive reasoning, evaluating how much performance improves from the base model when models are trained on data from different domains.", "questions": "When selecting domains, did you consider any others? If time allows, I think instruction following is a good verifiable domain to explore as well, and previous work has shown that models struggle to generalize to constraints beyond those they were trained on: https://arxiv.org/abs/2507.02833\n\nAlso, I'd recommend tweaking the citations in the first paragraph, right now there's essentially just a run on sentence of citations.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The results, while not incredibly surprising for those with substantial experience performing RL finetuning on language models, are quite valuable to see. The study is quite broad, only models with publicly available training data are included, and the experimental design is sound.", "weaknesses": "It would be helpful to list the models that you tested, both for reproducibility and for clarity. One question I have is how diverse the *base* model pool was; e.g. were most models based on Qwen (which is quite strong on math and code already), or was there a diverse set of model families included in your study?\n\nIf possible, it would be very enlightening if there could be a further study on the *kinds* of reasoning each model uses, to see if there are explicit strategies common amongst them (so we can better understand what \"tools\" models need for e.g. math or code), but this is mostly out of scope of this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ukzy12osuf", "reviewer": "Reviewer_FGBZ", "summary": "This paper looks at how reinforcement learning post-training generalizes to out-of-domain evaluations. The authors curate various models on huggingface trained with RL, and train their own models on specific domains. They find that RL post-training does not seem to generalize to arbitrary unseen domains, but does show generalization between math and code domains, which the authors hypothesise is due to similar reasoning templates being applicable in the two domains.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A wide range of models and evaluations are studied, making the findings seem fairly robust (especially with the intervention experiments).\n- The evaluation itself is well-performed, using appropriate metrics and statistical testing.\n- Paper is clear and reasonably structured, and the findings appear useful (especially that knowledge-focused tasks do not seem to transfer well to/from math and code settings).", "weaknesses": "- It appears that the open source models do not uniformly do better IID than OOD, for example, model 4 (Eurus-Prime) does 15 points better OOD than IID! Do you have explanations for this beyond ‘differences in implementation details’? It would be useful to have some idea of why there is this variance between models - the ID-OOD gap has a standard deviation of ~18 in table 2! I believe the trends described hold, but it would be good to have some idea of why there is this variance across models.\n- It seems that knowledge-RPT drops performance on the knowledge tasks - could this be more due to a domain mismatch between the knowledge-RPT data and evaluation tasks? The evaluations are very domain specific (medical QA, legal QA), while the training data is from multi-subject RLVR, which covers a much broader set of domains.\n- It would be useful to quantify/test the hypothesis in section 4.3 more thoroughly: if the reasoning templates are similar between code and math, could you examine some samples or measure overlap between reasoning chains to test this hypothesis? Looking only at downstream numbers does not fully explain what is happening. For example, it may be that only smaller subsets of the code data are similar to the math data, or that there is some cross-domain contamination between the two sets (e.g., code questions that require doing math, or math problems that require writing code).\n- For the intervention experiments, I’d be curious to see if the base model is a potential confounder. The deepseek distil model used has been extensively trained on math data, so it may be that this makes it less easy to adapt to knowledge tasks, or better primed to improve math performance when trained on code data.\n\nOverall, I think this is a solid paper, although its scope is somewhat limited. It would be useful to get more justifications around the knowledge-RPT setting and some discussion around the variance in the observational results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)