SELF-HARMONY: LEARNING TO HARMONIZE SELF-SUPERVISION AND SELF-PLAY IN TEST-TIME REINFORCEMENT LEARNING
Authors: Ru Wang
Wei Huang
Qi Cao
Yusuke Iwasawa
Yutaka Matsuo
Jiaxian Guo
Venue: ICLR 2026 Poster (2026)
Keywords: large language models
Test-time reinforcement learning
test-time adaptation
self-play
pseudo labeling
infomax
Primary area: foundation or frontier models, including LLMs
TL;DR: Through test-time self-play between a solver and a reframer, our method, Self-Harmony, uses an InfoMax-derived harmonic mean to score and select pseudo-labels based on their joint frequency across original and reframed questions.

Abstract: Test-time reinforcement learning (TTRL) offers a label-free paradigm for adapting models using only synthetic signals at inference, but its success hinges on constructing reliable learning signals. Standard approaches such as majority voting often collapse to spurious yet popular answers.
We introduce Self-Harmony, a framework built on a simple intuition: the correct answer should remain stable across both an original question and its paraphrase. Self-Harmony operationalizes this by employing a single model in two complementary roles: a Solver to produce answers and a Reframer to rephrase the input.  Based on this, we further propose a pseudo-label method: instead of majority voting, it aggregates answer frequencies across these original and reframed views using the harmonic mean. This is a process that naturally selects for solutions stable under reframing, thereby avoiding the common trap of favoring view-dependent, spurious answers.
Crucially, this requires no human supervision or auxiliary models. Across diverse reasoning benchmarks, Self-Harmony achieves state-of-the-art results at the label-free test-time setting, ranking first in 28 of 30 settings across multiple methods. Beyond accuracy, it demonstrates unprecedented robustness, with zero training failures in all experiments, underscoring its stability and reliability.

Published as a conference paper at ICLR 2026
SELF-HARMONY: LEARNING TO HARMONIZE SELF- SUPERVISION AND SELF-PLAY IN TEST-TIME REIN- FORCEMENT LEARNING
Ru Wang', Wei Huang?.;, 2.3, Qi Cao', Yusuke Iwasawa', Yutaka Matsuo', Jiaxian Guo 1 The University of Tokyo 2 RIKEN Center for Advanced Intelligence Project
3 The Institute of Statistical Mathematics 4 Google Research Australia q wang,
e wab.t.u-tokyo.co jp
wei huang, vrrriken. jp jeffguo@google.com
ABSTRACT
Test-time reinforcement learning (TTRL) offers a label-free paradigm for adapt- ing models using only synthetic signals at inference, but its success hinges on constructing reliable learning signals. Standard approaches such as majority vot- ing often collapse to spurious yet popular answers. We introduce Belf-Harmony a framework built on a simple intuition: the correct answer should remain stable across both original question and its paraphrase. Self-Harmony operationalizes this by employing a single model in two complementary roles: a Solver to produce answers and Reframer rephrase the input. Based this, we further propose a pseudo-label method: instead of majority voting, aggregates answer frequen- cies across these original and reframed views using the harmonic mean. This is process that naturally selects for solutions stable under reframing, thereby avoid- ing the common trap of favoring view-dependent, spurious answers. Crucially, this requires no human supervision or auxiliary models. Across diverse reasoning benchmarks, Self-Harmony achieves state-of-the-art results at the label-free test- time setting, ranking first in 28 of 30 settings across multiple methods. Beyond accuracy, demonstrates unprecedented robustness, with zero training failures all experiments, underscoring its stability and reliability. Our code is publicly available [Self-Harmony)
INTRODUCTION
Scaling the reasoning capabilities of Large Language Models (LLMs) has traditionally required mas- sive, cluman-curated datasets (Cobbe al. 202 Hendrycks et al.) 2021) for supervised finnituning (SFT). This reliance on costly data collection has motivated a shift toward test-time adaptation (Sun) al. 2020: Wang al. 2021), where models adapt "on the fly" using only unlabeled problems and available compute resources. Within this paradigm, test-time reinforcement learning (TTRL) has emerged particularly promising approach, enabling models to improve their reasoning by lever- aging self-generated feedback signals without the need for external supervision (Zuo et al. 2025
Zhang al. 2025 Prabhudesai et al. 2025 Zhao al. 2025b).
Current TTRL approaches typically involve generating multiple solution candidates and selecting
"aseudo-label" for the model learn from. often through self-consistency or majority voting (Wang
et al. 2023 Zuo et al. 2025 Liu et al., 2025a). However, this mechanism suffers from critical vulnerability: if the model exhibits a systematic reasoning flaw, may produce a specific incorrect answer more frequently than the correct one. In such cases, majority voting not only fails to correct the error but can actively amplify by selecting the flawed solution as a training target (Shi & Jin 2025 Huang al. 2024). This limitation highlights the need for TTRL methods that generate robust reward signals and reliably escape the model's own "echo chamber" of errors.
To address this challenge, introduce Delf-Harmony, novel TTRL framework that adapts self- play to generate reliable aseudo-labels in a fully self colf-contained manner. The core idea is grounded in the following intuition: the correct answer should appear robustly across two questions that

---

Published as a conference paper at ICLR 2026
semantically equivalent but stylistically distinct, as fragile spurious reasoning paths are often disrupted by changes in phrasing. Self-Harmony operationalizes this intuition by having a single model dynamically assume two cooperative roles: a Solver, which generates answers to the original problem, and a Reframer, which rephrases the problem to provide a diverse perspective.
Building this intuition, we observe that majority voting is not an ideal method for pseudo-label selection. Instead, we propose a pseudo-label generation strategy based on the harmonic mean of answer frequencies across both the original and reframed rollouts. This mechanism inherently re- wards answers that appear consistently in both distributions while penalizing those that arise only in one, thereby filtering out solutions driven by biased reasoning. With the pseudo-label selected by harmonic mean, the two roles engage in cooperative self-play: the Solver is trained to align with the robust pseudo-label, while the Refrumer is trained to generate informative reformu- lations that expose and challenge the Solver's current biases. This interaction enables the model to escape the limitations of majority voting.
We demonstrate the effectiveness of Self-Harmony through extensive experiments on multiple rea- soning benchmarks using open-source models ranging from 1.7B to 8B parameters. Our contribu- tions are threefold: (1) a novel TTRL framework, Self-Harmony, that adapts self-play to improve reasoning without labels or external models; (2) a robust pseudo-label selection mechanism based on the harmonic mean, which mitigates common failure modes of majority voting; and (3) state- of-the-art results across five open-source models and six challenging reasoning datasets. Notably, boosts the performance of Llama-3.1- 8B (Grtttafirri et al., (024) on GSM8K (Cobbe et al. [021] ) from 60.5% to 91.6%, and improves Qwen3-4B ( (Yang et 2025) on MATH500 (Lightman al. 2024) from 60.2% to 78.5%.
with only 16 original and 16 reframed rollouts, Self-
1.0
6.99
5:30
Symmbolic Style
Original Question
Wrong 120, 28,
0.8
Anown Consalency Soore
Calenbeting
Maiel
31 228
0.8
Correct
Question Semantie
Invariant
Changed Styln-Srodttive
0.4
Style--oaeessttivn
Cornect 4895
Reframed Question x"
0.2
Cuaike
alecr pue 30
unok
Maiik
ey da sosinds
0.0
Concrete Style
Wrong
Goode 35
Oren-th
ULMAS1
viie sa you chases thi
25,
(b)
Figure 1: (a) The illustration of correct answer consistency and wrong answer consistency of some Large Language models in the original question and semantic reframed question pair. We could see the correct answer consistency is significantly higher than the wrong answer consistency, which means if models answer the original question correctly, then it will be more likely keep the same answer asked by reframed question. (b) Illustration showing that the hypothesis that how question description style can influence model outputs. The original question & and its faithfully paraphrased counterpart a' share the same underlying semantic meaning but differ in surface style. A single model Modelg answers both questions, producing either correct or incorrect responses. This highlights the hypothesis that correct answers should remain stable across paraphrased views, while spurious errors are more dependent on stylistic variations.
RELATED WORK
2.1 SELF-IMPROVEMENT FOR LLM REASONING AT TEST TIME
A central challenge in advancing LLM reasoning is reducing reliance large-scale supervised datasets. Test-Time Reinforcement Learning (TTRL) has emerged as a promising paradigm, en- abling models to adapt during inference using only unlabeled problems (Zuo al. 2025 Zhang jet al. [025 Prabhudesai et al., 2025 Zhao et al.] 2025b). Many TTRL methods employ rule- based verifier provide learning signals. The most common intrinsic verifier is majority voting
2

---

Published as a conference paper at ICLR 2026
(or self-consistency) (Zuo al. 2025 Liu et al. 2025a), where the most frequent answer across multiple rollouts is selected as a pseudo- labuel for fine-tuning, While effective in some cases, this approach amplify systematic biases: if flawed reasoning process consistently produces the same incorrect answer, majority voting will reinforce that error with high confidence. Other meth- ods mitigate this issue by leveraging powerful external models for verification (Lightman et al.) 2024 Khalifa et al.) 2025 Zhao et al. (022a)), or employing external reward models to guide rein- forcement learning (Welleck al. [2024] Lambert al. 2024). However, such approaches violate the principle of fully self-contained, test-time setting. In contrast, Self-Harmony operates as pure TTRL framework, directly addressing the pseudo-label selection problem without relying on external supervision.
2.2 SELF-PLAY AND DATA AUGMENTATION
Self-play has been successfully adapted to LLMs, primarily in adversarial contexts where agents compete to identify and exploit vulnerabilities (c.g., "red-teaming") (Perez et al., 2022; Ge et al.)
2024
Kuba et al.) 2025). In contrast, work. Rather than competing, a single model assumes two collaborative roles-a Solver and a
work introduces novel cooperative self-play frame-
Reframer- that work together to produce solutions robust to changes in problem formulation (Xi et al.
2023 Zhou et al., 2024) Chen et al.) [024] Liu et al., 2024 Liang et al.] (2255] Zhang et al.) 2025 This reframing mechanism can be viewed as a form of online data augmentation, where
the objective is to generate diverse perspectives of a problem. The key intuition is that correct an- swers should remain invariant under such reformulations, whereas incorrect answers arising from fragile reasoning patterns will not be consistently reproduced. This cooperative dynamic- -imed distilling stable truths rather than exploiting weaknesses distinguishes our approach from prior adversarial self-play methods (Perez et al. 2022 Ge et al. 2024).
2.3 UNSUPERVISED SIGNAL SELECTION BY ENFORCING CONSISTENCY ACROSS VIEWS
At its core, Helf-Harmonyii is an unsupervised method that constructs its own supervisory signal with- out relying on ground-truth labels. Recent work uses confsconni as an unsupervised signal in reasoning and preference learning (Prasad al. 2024 Huang al. 2022), but these methods force consistency within a single view- tewn-tycccally via majority voting whereas we seek a stronger criterion that goes beyond simple vote accumulation. Our pseudo-label selection mechanism is spired by the information maximization (Infomax) principle, which encourages representations that remain invariant across different "views" of the data [Linsker 1988 Bell & Sejnowski] 1995 Hjelm et al.) 2019; Tian et al. 2020). This approach mirrors the principles of Invariant Risk Minimiza- tion (Arjovsky et al. 2019) and Group-Invarian Learning (Chen et al. 2020) by treating paraphrases as distinct environments, ensuring that selected answers are robust to surface-level transformations rather than relying on view-dependent spurious correlations. In our setting, the Solver and Reframer produce solutions to the original and reformulated problems, thereby creating two distinct distribu- tional views. To enforce agreement across these views, we employ the harmonic mean of answer frequencies, which provides stringent yet practical criterion (Xie et al., 2020 Sohn et al. 2020). Because the harmonic mean heavily penalizes low values, an answer must appear frequently in both distributions to be selected. This effectively filters out spurious solutions that arise from a single fragile reasoning path but fail to generalize under reformulation, enabling the model to construct a reliable, unsupervised learning signal.
METHOD
Adapting a pre-trained LLM to a targeted test set without ground-truth labels at test time requires the model to generate and reñne own training signal from uncertain rollouts. In this setting, pseudo-label selection becomes the bottleneck. To motivate our approach, we first formalize the problem and illustrate the limitations of prevailing pseudo-labeling strategies, and then introduce our core mechanism for generating dual-view consistency: using the model itself to paraphrase the original question and create a new, diverse problem perspective. Building on this, we provide a theoretical analysis showing that harmonic mean consensus is better pseudo-labelt view consistency. Finally, we introduce Self-Harmony, a practical algorithm that operationalizes this principle through structured self-play and robust policy optimization.
under multi-
3

---

Published as a conference paper at ICLR 2026
3.1 PROBLEM FORMULATION AND MOTIVATION
Given an unlabeled test dataset, for each question in the set, we aim to adapt our pre-trained LLM Ty to get a better performance on that test dataset. To achieve this, the model generates n rollouts for the input question pseudo-label y must be inferred from the rollouts and used as feedback for policy optimization. The quality of this pseudo-label directly dictates the stability of the adaptation process.
Since ground-truth labels are unavailable during test-time adaptation,
A dominant approach is majority voting: the answer most frequently generated by the model is se- lected. However, this simplicity comes at cost. When pCCorrect xx) < p(Wrong xt), Liu et al. (2025)) formally shows that as we draw more samples, the chance that majority voting recovers the correct answer actually converges to zero meaning that majority voting amplifies the wrong solution instead of correcting it. To resolve the "majority-vote trap" in LLM test time adaptation, we draw inspiration from a common human robustness heuristic: when confronted with uncertainty, people often check solutions across multiple perspectives or reformulate the problem to verify con- sisteney (Pólya 1945; Spiro et al. 1988). We hypothesize that the correct answer, even if not the most popular, should appear robustly across different, faithfully paraphrased versions of the question. Inspired by this, we argue that pseudo-labels should not be judged solely by popularity within single question description, but rather by invariance across multiple views. This motivates an alternative criterion based on information -theoortttttttteee principles.
3.2 THEORETICAL ANALYSIS: FROM MAJORITY VOTING TO HARMONIC MEAN
Our analysis begins by showing how a simple extension of majority voting to multiple views can still fail. We then introduce informationnthheorei objective grounded in view-invariance assump- tion. We demonstrate that the harmonic mean emerges as a principled second-orde approximation of this objective, providing robust mechanism for selecting pseudo-labels.
Building on prior research in dual-view approaches (Zhang et al., 2025; Federici et al. 2020), we extend the idea of majority voting to multi-view setting. Let Po(a) be the probability that the policy Te generates answer a for the original query x, and p1 (a) be the probability that the policy Te generates answer for reformulated query x'. A straightforward extension of the single-view majority voting principle is to aggregate the evidence from both views. This approach, which we term dual-view majority voting, selects the answer with the highest combined probability: y" arg max, (po(a) + P1 (a)). This method can resolve some single-view failures. For instance, if the correct answer C is less probable than a wrong answer W in the first view (Cn(C)) < Po (W)) but more probable in the second (Ci > Pı (W)). their sum may correctly favor C. However, dual- view majority voting remains vulnerable; it fails when different wrong answers dominate each view, as their summed probabilities can easily overwhelm that of the correct answer.
To overcome this limitation, we turn to an -nforriiion-theereiicc (Bell & Sejnowski 1995
Tsa
ct al. 2020 Hjelm al. 2019) approach guided by an assumption about the correct answers.
We view the correct answer C' as depending only on the underlying semantics of a query, not its superficial form. Thus, for two queries with equivalent meaning, the probability assigned C should remain unchanged. This motivates our core assumption below.
Assumption 3.1 (View- fiew-Invariance Assumption). For any two semantically equivalent queries x and x', the correct label C has an approximately constant probability mass: p(A = C' x) = p(A = C x²) where A is the random variable of answer In contrast, the probabilities of incorrect labels A + C vary across different views, reflecting their dependence on view-specific artifacts.
To formalize this, we define an objective that rewards accuracy while penalizing view-dependence. Let A be the random variable for the model rollout candidate answer a, X E {x, the view, and 2., = I = a} an indicator for whether the model's answer is a or not. We adapt the standard Infomax principle, which maximizes the mutual information I [(Zo; A), by introducing a penalty term (2.;; X) that discourages dependence on the specific view. This yields the View-Invariant Infomax objective:
[x 1(2:; -1 XX)
where the hyperparameter À balances the trade-off between accuracy and view-invariance.
4

---

Published as a conference paper at ICLR 2026
Theorem 3.2 (Harmonic Mean Selector from Invariant Infomax). Assume the view-invariang dition (Assumption 3.1 holds. Suppose moreover that the following conditions are satisfied.
Al. Nonddegenrr.cy. For every label po(a) + Pi(a) 1. (This simply excludes the trivial case in where the majority voting can solve.)
A2. Balanced Coneedence. There exists constant (0,1) such that for every maximiser a' of Jx(+) (po(a") pi(a")) N ( pi((a")
A3. Uniform View Prior. The view variable X E { is assumed to be sampled uniformly, i.e., p(X z) -((X x') 3.2 This assumption ensures that the penalty term I [Zai reflects dependence on the view itself rather than bias in the sampling distribution.
Then, for the penalty weight X = 2, the pseudo label that maximises the second approximation of the view-invaring Infomax objective J2(a) = I [ZZ:: A) - (Zo; X) is obtained by the harmonic mean of the two view-probabilities:
y arg max 2po p1 (a
E arg max J2(a)
po(a) (a)
The full proof can be found in Appendix G.I Theorem 3.2 implies that the harmonic mean is not an arbitrary heuristic but principled regularizer that enforces view-invariance. naturally emphasizes answers that maintain consistent support across views while aggressively penalizing those that strong in one view but weak another. This aligns well with the intuition that a true answer should be robust reformulation, while spurious artifacts should not.
3.3 Self-Harmony FRAMEWORK
Policy Optimization
(st)
Rewards (ri) of (s)
4495
28
28
4495
Hide Ywilher
GIUO Oltjective
Calesiing
28
28
0
Ours: Harmonic Mean Pseudo Label
Retromer
2/sto1yp.oo mux 2001-91001
Pre-Traked
Caksliz stecc survec caskk freqpence [nd] Bein!
4485 /
panmelees
dac''s frequencs Isch
Baseline: Majority Voting Pseudo Label
Misy
Refeaned
Cunakar the accaato vtalco rpecii grapp eveyy (ifiven ways you daese gloup
20
120
Mude Verther
GRIO Obgeetive
4495
4495
Rewand(r)) of (s
Extracted Rollout Answers (x() of
Figure 2: An overview of our framework. Given an unlabeled question æ, the policy model Te will paraphrase it to new question z', and then generate sets of extracted rollout answers {v, and [v(} respectively. rule-based verifier assigns reward to each response based on its correctness against target y" got by harmonic mean instead of majority voting. These rewards are aggregated into objective functions, and J', which guide the optimization of the pre-trained LLM.
Based on our theoretical analysis which concludes that View-Invariant Infomax pseudo-labeling rule for robust multi-view adaptation is to select answers based on their harmonic mean score, The Self-Harmony framework the practical instantiation of this principle. operationalizes this by having single model engage in structured self-play. The model generates answers from two distinct perspectives-a2 original view and a self-generated, reframed view. The harmonic mean consensus rule then identifies the most consistent answer across these views serve the training signal, ensuring that only perspective-stable solutions are reinforced.
Single Model, Dual Roles for View Generation. Self-Harmony switches between two roles via prompting. The Solver role ग. generates answers to query æ, while the Reframer role po creates semantically equivalent paraphrase, x'. The framework follows
single model (Ma) that
5

---

Published as a conference paper at ICLR 2026
intuitive "solve - reframe + solve" conceptual sequence. First, the Solver To generates answers for æ. Second, the Reframer Pe generates x'. Third, the Solver Te is invoked again to generate answers for x'. For computational efficiency, practical implementation optimizes this process; we detail this in the Experiments section.
Pseudo-Laie Label Selection via Harmonic Mean Score (HMS). Following our theoretical analysis (Theorem 3.2). we select the pseudo-label y" by identifying the candidate answer a that maximizes the Harmonic Mean Score of its empirical support from the original (a(a) and reframed Pr (a) views
according: y² arg maxe Pu(a)+Pi(a)
(po(a)pn(a)
3.4 FROM CONCEPTUAL FRAMEWORK TO PRACTICAL IMPLEMENTATION
Algorithm 1 Helf-Harmony (Conceptual Flow)
Input: Language Model Ms. Test Datnset D. Number rallouts
Deline: Training Step Problem Solver mas Reframer Ps. They shared the parameters
3: Output: Adapted model parameters 0r
4: for (= to T do
5: Sample minibatch
for query E do
Generate extmacted rollout answer (y) via g - Tol: (a) Solver on
8:
Generate zf - pe(: .r) // Refrrme question from 10.x"
Generate (y')
mu(- (x)) Solver on
10:
Calculate A(euch answer candidate)'s frequency in [1.]: folo) and in (vd): (i(a)
11:
Compute the pseudo-label y² HM)Sppoe)) (1))))
12:
Compute rewards: Rusts. Flave and diversity reward Rittae
Compute ((( ())
rewards
14:
Update by ascending VJ((0) (.)
15:
end for
16:
end for
The core principle Self-Harmony is a three-step solve - reframe + solve sequence, as outlined in Algorithm [ This conceptual flow isolates three distinct actions: initial solving action the original query, a reframing action, and final solving action on the paraphrased query.
However, executing this sequence with three separate model calls is computationally inefficient. To create a practical and scalable algorithm, we optimize this flow by fusing the reframing and the second solving steps into a single, structured generative action. This implemented in practice using a system prompt that instructs the model to first paraphrase the question and then immediately solve its own paraphrase within a single generation (see Appendix D.I full prompt). This optimization reduces the process to two efficient model calls: one for the initial solution and one for the joint "reframe-and-solve" trajectory.
3.4.1 POLICY OPTIMIZATION WITH FUSED ACTIONS
This fusion fundamentally changes the optimization problem. Instead of three actions, the policy now executes two, requiring reward structure that reflects this new dynamic.
Reward for the Initial Solving Action. The first solving action remains unchanged. It is executed by the policy Ty and rewarded for matching the pscudo-label y", which is determined after both trajectories have been generated. The reward is simply the correctness of the generated answer y: Reoive (y) = TI = y*].
Reward for the Fused Beframing-and-Solvve Action. The composite action requires a reward that jointly evaluates paraphrase quality and final-answer correctness. A standard additive reward is suboptimal, as it grants partial credit for a well-formed paraphrase even when the final answer is wrong (see Appendix E.5)
We instead design a reward where answer correctness acts as a success gate. A base reward is given only when the answer is correct and is modulated by two penalty terms: Format Penalty (espenalty ) for structural violations, and a Diversity Penalty (Fzpenally), ). which is the Jensen-Shannor divergence between the original and reframed queries' answer. Here, each (1 w Rpenalt) term acts as reward, so higher penalties reduce the final score. The final reward is defined as Riweed(')) =
(1 Wy ppeenally (v/) (1 w. Pepeaalyy (u', Ty) y² where Tly v*] is if the predicted answer

---

Published as a conference paper at ICLR 2026
correct and 0 otherwise. This ensures that only correct receive reward, while the penalties shape the signal to favor reframings that are both well-formed and meaningfully different.
Main results on reasoning benchmarks. Self-Harmony consistently outperforms base- line methods across most tested models and datasets, demonstrating both superior performance and training stability. The best and second-best results are highlighted. Specifically, a baseline's per- formance degrades significantly after its peak, we report the highest score observed across validation steps and mark with an asterisk (*) while ours Peff-Harmoyy use the score final step.
Table 1:
Mathematies
Multi-Subject
Multi- Task
Methods
MATH500
GSM8K
AIME 2024
AMC
GPQA
MMLU-Pro
Qwen3-1.7B-Ban
Before RL
65.58
3.33
26.50
20.30
16.61
GT-Reward
71.80
85.97
20.83
53.01
53.80
85.71
Intuitor
51,12
80.25*
3.75*
23.56*
23.76
31.25
Rent
78.64*
6,45*
32.00*
23.47
Majority- Voting
64.64
83.80
9.37
37.65
24.68
44.82
Ro-RRwarrd
39.75
23.66
47.14
Self-Harmony
87.47
40.51
27.92
53.66
Qwen3-4BB
--een-3-BBBasee
Before RI.
60.20
55.72
34.94
16.75
27.59
GT-Reward
86.74
92.89
92.85
Intuitor
72.35*
87.53*
10.41*
43.59*
32.39*
64.55*
Rent
74.60
90.49*
12.08
45.25*
31.78*
66. 16
Majority- Voting
75.75
93.44
20.00
49.32
36.51
52.95
-Co-Rearrd
76.54
93.47
12.71
46.98
24.36
51.79*
Self-Harmony
78.50
94.31
20.00
49.40
37.06
67.68
Qwen3-8B-Band
Before RL
66.80
84.76
10.00
45.78
33.44
50.09
GT-Reward
84.72
96.51
56.66
87.95
93.40
92.85
Intuitor
78.77"
92.28
17.08*
51.65*
31.56"
60.54
Rent
77.26*
91.20*
18.54
50.82*
37.88*
69.91
Majority- Voting
78.99
94.00
24.16
59.03
39.34
77.58
Co- Reward
78.92
94.80
15.83
31.80
24.42
$7.59*
Self-Harmony
80.00
95.45
23.33
59.04
38.07
77.68
Llama-3.2-33 3.-sstrtt
Before RL
39.80
16.65
6.67
19.27
3.04
34.11
GT-Reward
69.80
93.70
33.33
83.13
89.34
72.85
Intuitor
4
16.73*
9.16*
23.79*
3,23*
34 64.
Rent
47.30*
16.67*
7.70*
23.79*
2.79*
34.38*
Majority- Voting
46.38
85,98
13.33
20.48
22.33
31.43
Co- Co-Reward
55.22
89.14
13.33
25.30
19.79
34.02*
Self-Harmony
55.40
89.55
13.33
25.30
29.95
44.29
1lama-3.1-88 3-8---nsttuctt
Before RL
41.46
60.48
3.33
20.48
14.72
43.75
GT--Reward
73.60
95.60
33.33
80.72
89.84
91.42
Intuitor
48.37
66.25
6.45*
6.62*
15.32*
40.00
Rent
45.98
69.88*
3.95*
21,98*
14.56*
40.80%
Majority-Vonn Votting
46.71*
88.78*
4.13
21.53
25.31
45.36*
Co-Reward
48.01
89.48
3.33
17.84
24.36
42.77
Self-Harmony
50.40
91.59
10.00
26.51
28.93
50.00
EXPERIMENT
4.1 SETTINGS
Benchmarks and Baselines. MATH500 (Hendrycks et al. 2021), GSM8K (Cobbe et al, 202I), AIME 2024, AMC, GPQA- Diamond (Rein et al. 2023). and MMLU-Pro (Wagg al. 2024). Following (Zuo et al. 2025
We evaluate method diverse set of reasoning benchmarks:

---

Published as a conference paper at ICLR 2026
report the average pass @ over 16 rollouts. Our primary baselines recent label-free meth- ods: Intuitor (Zhao et al. 2025b), Rent (Prabhudesai et al. 2025), TTRL (Zuo et al. 2025). and Co-Rewart (Zhang et al. 2025). For Co-Reward, match model sizes for question rewriting rather than relying on Qwen3-32B. Specifically, Qwen3-XB-Bal uses its corresponding instruct model, and Llama-Instruct uses its For rewriting prompts, follow the settings described in the original papers. To ensure fairness, all methods use the same hyperparameters and official implementations.
Models and Training Details. To demonstrate the generality of our approach, we evaluate across range of open-source models: Owen3-1.7B-Base, Qwen3-4B-Base, Qwen3-SB-Base (Yang al. 2025). Llama-3.1 1-8--nnstrtttt (Grattafiori et al. 2024), and 1aama-3.2-BB-Instrue
All experiments use learning rate of 10 "with a cosine schedule. For each dataset, we adopt fully unsuper- vised reinforcement learning setting: separate model is trained on the dataset without using any ground-truth labels at any stage. Although the same dataset later used for evaluation, correctness labels are never used to generate pseudo-labels or to form any training signal, so no correctness information leaks into training. All pscudo-labels during training are produced exclusively by the model's 's own rollouts, and every result reported corresponds to model independently trained on that dataset under this strictly label-free setting. A complete list of hyperparameters and experimental settings is provided in Appendix D.2
Fair Comparison Computational Budget. Multi-view methods incur additional computa- tional cost since our framework generates answers from two views. To ensure fairesss, all baselines were allocated a comparable budget. Specifically, if Self-Harmony uses Nr rollouts for both the original and reframed views (totaling 2N,),, then each baseline also allowed 2.N, rollouts from its single view. This setup guarantees that performance improvements are attributable to method- ological effectiveness rather than increased computation.
4.2
MAIN RESULTS
We evaluate Self-Harmony against several baseline reward modeling techniques across five rea- soning benchmarks using five base models from the Qwen3 and Llama-3 series. The results are summarized in Table
Dominant Performance Across Benchmarks. Across 30 configurations (5 models x bench- marks), Self-Harmony ranks first in 28 cases and second in the remaining 2, demonstrating strong generalizability across both model families (Qwen, Llama) and parameter scales (1.7B-8B). For example, on MATH500 with Qwen3-1.7B, accuracy improves from 42.70% 69.60% while no baseline exceeds 65%, nearly matching the oracle GT-Reward score of 71.80%.
Merr600 Passo
Swil-Harmont
CoReward
ReNT
Co Reward
02
Sall-Hamony
60 62 X0C
180
(a) Training Stability
(b) Pseudo- labell accuracy on Math500-Lever
Figure 3: (a) Self-Harmont demonstrates the highest training stability with Llama-3. 1-8--Insttruct on MATH500. (b) Comparison of pseudo-label accuracy. Self-Harmony consistently generates pseudo-labels with the highest accuracy throughout training, significantly outperforming Co-Reward and TTRL. For Ce-Reward, we use the average accuracy for both original branch and reframed branch.
Superior Training Stability. As shown in Table I Fig. 3a Self Helf-Harmony remains stable across all experiments, highlighting the robustness of its reward modeling and training framework.
8

---

Published as a conference paper at ICLR 2026
4.3 ANALYSIS
In this section, we conduct detailed analysis to validate the core mechanisms of our Self-Harmony framework. We first analyze our Harmonic Mean Score (HMS), showing that effectiveness in generating pseudo-labels corresponds logically with problem difficulty. We then present a compre- hensive ablation study that confirms each component our reward objectives and our pseudo-labe selection strategy-is critical to the model's success. Col- lectively, these findings validate the key design principles and overall effectiveness of our approach.
the Reframer's
4.3.1 PSEUDO-LABEL QUALITY EVALUATION
The success of Self-Harmony is driven by the comprchensive quality of its pseudo-labels. We evalu- ate this quality not only through raw accuracy but also through stability and calibration metrics. As shown in Figure b), our method achieves the highest accuracy MATH500 (Level 3), stabilizing around 60.85% To further elucidate why our approach excels, we analyze additional signal prop- crties-including Fl Fl-score and Spearman correlation-in Appendix L These metries confirm that harmonic mean score provides more calibrated supervision by penalizing spurious, siew-dependent answers.
42
0.943
0.952
0.962
0.352
0.952
0.960
Format Coefficient
-0,855
84
0.952
0.952
0.943
0.952
0.943
0.90
Pass
Pass
0.85
0.945
0.952
0.943
0.952
0.943
0.952
0.80
Mean Pass@1
0.933
0.943
0.962
0.943
0.943
0.75
Varied Range
-0.935
0
20
40
60
80
100
Diversity Coefficient
Step
(a) Performance Heatmap
(h) Performance Range
Figure 4: Sensitivity analysis for the format (wy) and diversity (Wa) reward coefficients. (a) A
heatmap showing consistently high performance across wide range of coefficient values. (b) A
plot illustrating the uncertainty range, confirming the model's robustness to hyperparameter choices.
4.3.2 MODULE ABLATION STUDY
Table 2: Ablation study on the MATH500 benchmark. We analyze the impact removing the Format and Diversity rewards from the Reframer, and replacing our Harmonic Mean Score (HMS) based Pseudo-label Selection with simpler Majority Voting and Cross selection methods. All results are averaged over the last three evaluation checkpoints.
Math500
Self- Harmony (Full)
Reframer
Psendo-label Selection
w/o Format Reward w/o Diversity Reward
Cross Selection
Majority Voting
Qwen3-4l-BBan Qwen3-8hhBawe
78.50
78.40
78.20
76.50
77.30
79.80
77.46
78.90
78.40
79.00
conduct ablations to evaluate each component's contribution (Tab. 2.. For the Reframer, re- moving either the diversity rewardor the format reward consistently degraded performance. On Qwen3-8B, removing the diversity reward reduced accuracy from 79.80% to 78.90%, showing that second view must be meaningfully different to be effective.
For Pseudo-label Selection, we compare our HMS with two simpler strategies: (1) Majority Voting, where branchs uses their majority-voted outputs, and (2) Cross Selection, where branches swap outputs. Both underperform HMS, with Qwen3-8B dropping to 79.00% (Majority) and 78.40% (Cross) 79.80% for HMS.
These trends hold for both 4B and 8B models, confirming the necessity all core components for Self-Harmony' effectiveness.

---

Published as a conference paper at ICLR 2026
Original Question
Reframed Question
A path is described by the equation y 4y How many points this path touch the vertical line at 0)
How many u-intercepts does the graph the parabola x = y² 4g have?
Original Question
Reframed Question
Imagine you're a craftsman dividing 2- foot-long wooden board smaller, pre- cise segments. You notice that you have a 3-inch piece that you need.
What fraction 2 feet inches?
Express your answer as a common frac- tion.
What fraction of 2-foot board does this 3-inch piece represent?
Figure 5:
Examples of original and reframed questions. The Reframer rewrites the problem while preserving its semantics and ensuring the correct answer remains unchanged.
4.3.3 SENSITIVITY TO REWARD COEFFICIENTS
Our Reframer's reward is governed by two key hyperparameters: the format reward coefficient Wy and the diversity reward coefficient wa- To assess the sensitivity of Self-Harmony to these parameters, conducted grid search, varying both coefficients from 0.2 1.0.
The results, visualized in Figure 4 demonstrate that Self- Harmony robust to the choice these coefficients. As shown in the heatmap (Figure 4aj, the model's performance on Math500 remains high and stable across the entire grid, with accuracy scores consistently above 89.5%. This stability a significant advantage
indicates that our method does not require extensive or precise hyper- parameter tuning to achieve strong results. Figure 4b) further illustrates this robustness by showing the minimal uncertainty in performance across different settings.
4.4 REFRAMED QUESTION EXAMPLES
Figure shows that the Reframer rewrites the problem introduce stylistic diversity while rigor- ously preserving the underlying semantic constraints. This confirms the module's 's ability to perform semantic-invariant transformations, ensuring that the diversity needed for consistency checking is generated without altering the problem's logic. Additional examples are provided Table 6 and Table 7 Appendix F
CONCLUSION
Effective test-time reinforcement learning (TTRL) is hindered by the challenge of robust pseudo- label selection, as standard methods like majority voting often amplify an LLM's inherent reasoning flaws. In this work, we introduced Self-Harmony, that such flaws are often fragile and tied to specific problem phrasings. Through self-play mech- anism, single model acts as both a Solver and Reframer to generate second, paraphrased view of each problem. We show theoretically that the Harmonic Mean Score (HMS) is robust criterion for identifying the invariant answer across these views, rewarding solutions that are stable under rephrasing. This approach achieves Mlate-of-the-ant performance in extensive experiments. Notably, across 30 configurations, Self-Harmony ranks first in 28, demonstrating strong generalizability. For example, MATH500 with Owen3-1.7BB 1B, while no baseline exceeds 65%, nearly matching the oracle GT- Re-Re score of 71.8%.
label-free framework built on the core insight
improves accuracy from 42.7% to 69.6%,
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our work focuses on improving the reasoning capabilities of large language models through label- free, self-correction mechanism. We primarily use publicly available and widely-cited academic benchmarks (MATH, GSM8K, AIME, AMC, GPQA) and open-source models (Qwen3, Llama- 3 series), avoiding the collection of new, sensitive, or personally identifiable data. Our method, Self-Harmony, does not introduce new sources of societal bias; however, it relies foundational models which may inherit biases from their training data. While our research aims to advance the reliability of Al reasoning, generally beneficial goal, we acknowledge that any advancement in Al capabilities carries dual-use risk. We have not identified any direct negative societal impacts or ethical concerns stemming specifically from our proposed method. All research conducted with a commitment to academic integrity.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, we have made our implementation publicly available. The complete source code for Harronyhonny/ link in Appendix (A), along with scripts to repli- cate the experiments reported in our paper, is included in the supplementary materials. Our ex- periments are conducted exclusively on publicly available datasets, including MATH500, GSM8K, AIME, AMC, and Base, 3-aam--..--8---- I-8B-IInstruct are clearly cited in the main text. Detailed experimental settings, including all hyperparameters, prompt templates, and specific versions of the models, are docu- mented in the appendix to facilitate replication. Our evaluation metric, pass@ 1, is standard for these benchmarks, and our comparison with baseline methods uses identical learning hyperparameters to ensure fairness.
The specific open-source models used (e.g.. Qwen3-1.7B.
REFERENCES
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint
2019.
Anthony J Bell and Terrence J Sejnowski. An information-maxxiizzaion approach to blind separa- tion and blind deconvolution. Neural computation, 7(6):1129-115,, 1995.
Shuxiao Chen, Edgar Dobriban, and Jane H Lee. A group-theoretic framework for data augmen- tation. Journal of Machine Learning Research, 21(25)):1-71, 2020. URL httpp://ww// //www. www.r.org/pppeee
20- 1t3.htll
Wenqing Chen, Weicheng Wang, Zhixuan Chu, Kui Ren, Zibin Zheng, and Zhichao Lu, Self- para-consistency: Improving reasoning tasks at low cost for large language models. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Compu- tational Linguistics: ACL 2024, pp. 14162-14167, Bangkok. Thailand, August 2024. Associ- ation for Computational Linguistics. doi: 10. 10.18863/11/2224400 http:////......inaaaaaalllll sindings-acl.842. URL https: hellantholloy..or//22200
8427
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Henryk Michalewski, Jacob Hilton, Reiichiro Nakano, Christo- pher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 14.68 2021.
Marco Federici, Anjan Dutta, Patrick Forré, Nate Kushman, and Zeynep Akata. Learning robust representations via multi-view information bottleneck, 2020. URL
abs/2002.07017
Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. MART: Improving LLM safety with multi-round automatic red-teaming. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies ( (Volume I: Long Papers), pp. 1927-1937, Mexico City, Mexico, June
11

---

Published as a conference paper at ICLR 2026
2024. Association for Computational Linguistics. doi: 10.18633/1/22224macccc URL
httpss//aelanttooooo.oroo/////// wttp:////g/22...aaanooooooo..
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshor, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Ko- renev, Arthur Hinsvark. Arun Rao, Aston Zhang, Aurelien Rodriguez. Austen Gregerson, Ava Spataru. Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruy Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Eytimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadcokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak. Jong- soo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yearr, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Mclanie Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng. Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan. Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Kam,, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do. Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Gold- schlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg. Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An- nie Dong. Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman. Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang. Beth Loyd, Beto De Paola. Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic. Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang. Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Elconora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le. Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia
12

---

Published as a conference paper at ICLR 2026
Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi. Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Hari- son Rudolph, Helen Suk, Henry Aspegren. Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Flena Veliche, Itai Gat, Jake Weissman, James Gehoski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh. Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang. Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang, Rachad Alao, Rachel Ro- driguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang. Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan. Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen. Steve Kehoe, Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang. Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Ku- mar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang. Xiao- jian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Qi. Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhao- duo Wen, Zhenyu Yang. Zhiwei Zhao and Zhiyu Ma. The llama 3 herd of models, 2024. URL o abs/ 2177.2178
titpps:/aaze
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv 2021.
preprint
R Devon Hjelm, Alex Fedorov, Samuel Davoic-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estima- tion and maximization. In International Conference on Learning Representations, 2019. URL htttss://ppereviii
.et. /tt///ommmndB111300rrrm
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. arXiv preprint (rXiv:2210.11611 2022.
Siyuan Huang, Zhiyuan Ma, Jintao Du, Changhua Meng, Weiqiang Wang, and Zhouhan Lin. Mirror- consistency: Harnessing inconsistency in majority voting. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP Miami, Florida, USA, November 2024. Association for Computational
2024, pp. 1408-2420 Linguistics. doi: 10. 10L11886611/0/ www.:niiiiinng-emn...33.. URL org/2024. 1.234..inne findings emnlp 1357
Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jackyeom Kim, Hao Peng, Moon- tae Lee, Honglak Lee, and Lu Wang. Process reward models that think. arXiv preprint arXiv:2504. 16828, 2025.
13

---

Published as a conference paper at ICLR 2026
Jakub Grudzien Kuba, Mengting Gu, Qi Ma. Yuandong Tian, and Vijai Mohan. Language self-play for data-free training. 2025. URL https: http:://aaxxvvoooo.. http://t.......////////////// www.aa/........... 07414
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, Lester James Validad Miranda, Bill Yuchen Lin, Khyathi Raghavi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language model- ing. arXiv preprint arXhv:2403.13778 2024.
Xiao Liang, Zhongzhi Li, Yeyun Gong, Yelong Shen, Ying Nian Wu, Zhijiang Guo, and Weizhu Chen. Beyond pass@1: I: Self-play with variational problem synthesis sustains rlvr, 2025. URL
https://arxii www.orgggss... abs/ 2508 14029
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2024.
net net//orrumIid-v8100NN0
R. Linsker. Self-organization in a perceptual network. Computer, 21(3):105-117, 1988. doi: 1109/2.36.
Jia Liu, ChangYi He, YingQiao Lin, MingMin Yang, FeiYang Shen, and ShaoGuo Liu. Ettrl: Bal- ancing exploration and exploitation in Ilm test-time reinforcement learning via entropy mecha- nism, 2025a. URLhttts::
arxiv. orq/ abs 2200. 11356
Qin Liu, Fei Wang, Nan Xu, Tianyi Lorena Yan, Tao Meng, and Muhao Chen. Monotonic para- phrasing improves generalization of language model prompting. In Yaser Al-Onaizan, Mo- hit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Lin- guistics: EMNLP 2024, pp. 9861-9877 Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 186555////22....ninnnnemmm////// findings-emmlp.577. URL https:
loglantholooy.. wwwpatthhooooy..og/222222.2 findings emn 1 1ommmlpp11 5767
Yexiang Liu, Zekun Li, Zhi Fang. Nan Xu, Ran He, and Tieniu Tan. Rethinking the role of prompting strategies in Ilm test-time scaling: A perspective of probability theory. arXiv preprint arXiv:2505.10991, 2025b.
Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3419-3448, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 1855/v 1/2022. emnlp-main.225. URL /ttl:ttts///aattttttttt
ac aclanthology org//022 emnlp ma iiainn.2 225/
Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Maximizing confidence alone improves reasoning. 2025. httpss/tts://e abs//
2505.222 22660
Archiki Prasad, Weizhe Yuan, Richard Yuanzhe Pang, Jing Xu, Maryam Fazel-Zarandi Mohit Bansal, Sainbayar Sukhbaatar, Jason Weston, and Jane Yu. Self-consistency preference opti- mization. arXiv preprint 1:.Xi:2411.041009 2024.
George Pólya. How to Solve It. Princeton University Press, 1945.
David Rein, Stas Gaskin, Llion Jones, Vamsi Aribandi, Yi Tay, Dara Bahri, Daniel Mendoza, Ekin Akyurek, Kellie Fusco, Chen Wu, Ja- Young Sung, Sebastian Gehrmann, Izzeddin Gur, Naman Goyal, Srivatsan Ajay, Jonathan Clark, William W. Cohen, and Slav Petrov. Gpqa: A graduate- level google-proof q&a benchmark. arXiv preprint arXiv:2311.
2023.
Wenlei Shi and Xing Jin. Heimdall: test-time scaling on the generative verification. arXiv preprint arXiv:2504. 10337, 2025.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.
14

---

Published as a conference paper at ICLR 2026
Rand J. Spiro, Richard L. Coulson, Paul J. Feltovich, and David K. Anderson. Cognitive flexibil- ity theory: Advanced knowledge acquisition in ill-structured domains. Proceedings of the 10th Anmual Conference of the Cognitive Science Society, pp. 375-383, 1988.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time train- ing with self-supervision for generalization under distribution shifts. In International conference on machine learning, pp. 9229-9248. PMLR, 2020.
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in neural information processing sys- tems, 23:6827-6839, 2020.
Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, and Louis-Philippe Morency. Self- supervised learning from a multi-view perspective. arXiv preprint N0Kiv:20006005576, 2020.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. International Conference on Learning Repre- sentations, 2021. URL
n
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL bttts:://ppereevi/
net Bett:/ooummmdd-PIIIN
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj. Xuan He, Ziyan Jiang. Tianle Li, Max Ku, Kai Wang. Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark, 2024. URL
Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. From decoding to meta-generation: Inference-time algorithms for large language models. arXiv preprint arXiv:2406.16838. 2024.
Zhiheng Xi, Senjie Jin, Yuhao Zhou, Rui Zheng, Songyang Gao, Jia Liu, Tao Gui, Qi Zhang, and Xuanjing Huang. Self-Polish: Enhance reasoning in large language models via problem re- finement. Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNIP 2023, pp. 11383-11 1406, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 118666/31/2222 hag.:////m//2222.nnnnnnneeemmmm findings-emnlp.762 URL 200./2223..co findings wttp:///woorr/222.......e.gg.oo emnlp. 1m.11.76211
Qizhe Xie. Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33:6256-6266 2020.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng. Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URI. htt.tttps////
org/ abs / 2505.0.. 09388
Zizhuo Zhang, Jianing Zhu, Xinmu Ge, Zihua Zhao, Zhanke Zhou, Xuan Li, Xiao Feng, Jiangchao Yao, and Bo Han. Co-reward: Self-supervised reinforcement learning for large language model reasoning via contrastive agreement, 2025. htLlhttp:///rret www.ivoorg//o..ooooooooo abs/2508.
00410
Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, and Bowen Zhou. Genprm: Scaling test- dest-time compute of process reward models via generative reasoning, 2025a. URL httl:/tps//aanc..orr//bb////////
15

---

Published as a conference paper at ICLR 2026
Xuandong Zhao, Zhewei Kang. Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards, 2025b. URLhtttp::/arxxi. http://ww.rrgab/////50500/ 199.00.999900 19590
Yue Zhou, Yada Zhu, Diego Antognini, Yoon Kim, and Yang Zhang. Paraphrase and solve: Ex- ploring and exploiting the impact of surface form on mathematical reasoning in large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Con- ference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 1: Long Papers). pp. 2793-2804, Mexico City. Mexico, June 2024. Association for Computational Linguistics. doi: 10. 1211185/11/22244 htw.///002..aacloonngg.g 153. URL. antho hww.hhooooooorogo222ooee nta.44.aacllleel long. 1537
Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. Ttrl: Test-time reinforcement learning. arXiv preprint arXiv:2504. /6084, 2025.
16

---

Published as a conference paper at ICLR 2026
A CODE
The code is publicly available athttps: low the instructions to run our code and reproduce our results.
htth::::/hhhtttssstihttieoei httt://ggthbb... http://byy..csrrseee_eaaaaaaa Fol-
B VISUALIZATION
B.1
PSUEDO LABEL SELECTION
Value
IDS
[x]:
[aaas ssoo]
4495
0.250
0.250
Silect
use
28
0.300
0.125
28
0.200
y' 4495
(/)
4405
120
0.000
0.575
120
0.000
Figure 6: Pseudo label selection with pre-computed harmonic mean values.
The psuedo label selection is like following and shown Figure 6 counting, calculating the fre- quency, calculating the harmonic mean score, picking the candidate with largest harmonic mean as pseudo label.
B.2 FUNCTION OF DIVERSITY REWARD
Original
b= &bbc - -6, what is the value of the product ode?
Low diversity reward
High diversity reward
Nearly Overbapping
Distinct distributions
Small Modification
Good Rewrite
You of equations with three varisbles: a, equations are:
Imagine magical land where there ilroe mystical creatures, each unique power. Elf. Gnome, and a Troll The strength of their combined powers follow these ancient laws: The Eil and the Gnome together
3
combined power
and the Troll together have combined power of -3 unibs. The Elf and the Troil together have combined of units.
Your task find of the product obe solving system equations cach pair of variables adis up to the given values.
product wizard has discovered spell o,
powers, and represent the power levels the EIr, the Gnome, and the Troll, respactively.
The impact of the diversity reward the Reframer policy. Without strong diversity signal, the model may produce trivial modifications (x') that fail test robustness. The diversity reward, based model to generate creative and semantically rich reformulations (z") that provide a more challenging and effective consistency check.
Figure 7:
the Jensen-Shannon Divergence between answer distributions, incentivizes the
The diversity reward incentivize the reframer to reframe question in a non-tie non-trivial way with distri- bution distance, shown in Figure
17

---

Published as a conference paper at ICLR 2026
C
PSEUDO LABEL CROSS DIFFERENT DIFFICULTIES
Pseudo Label Math Pass@1
0.7
0.8
0.5
0.4
0.3
20
40
60
80
100
Step
Figure 8: Pseudo-label accuracy generated by our Harmonic Mean Score over 100 training steps, categorized by math problem difficulty (LI easiest, L5 hardest). The accuracy shows clear correlation with problem complexity, achieving high performance levels L1-L3.
D IMPLEMENTATION DETAILS
D. PROMPT DESIGN
We design two distinct prompts for the Solver and the Reframer components.
During both training and evaluation, the Solver is guided by straightforward instruction- following prompt to promote clear, step-by-step reasoning:
Solver:
Let's think step by step and output the final answer within \boxedi:
Reframer. The Reframer tasked with creatively rephrasing the original problem before solving it, promoting diversity and robustness. The prompt explicitly instructs the model to first transform the problem using one several strategies, then solve transformed version. The transforma- tion must preserve logical and numerical consistency with the original problem, while discouraging trivial paraphrasing. The full prompt shown below:
You are The Reframer, an that sclves problems with creative flair. Your challenge is not just to find the answer, but to find a more interesting way to get there.
Mission:
1. Transform: NOT solve the ORIGINAL PROBLEM directly.
Rewrite it using one of the strategies below to unlock a new perspective.
2. Solve: Sclve the new problem you created. Your solution's style must match your chosen transformation.
Strategies (Toolkit):
Concretize: Turn an abstract idea into tangible story.
Generalize: Convert specific case into a general formula.
Domain Shift: Change the domain (e.g., math -> code, logic -> game).
Add Noise: Introduce red herrings to test focus. Reverse: Start from the final result and work backward.
Incremental Complexity: Solve simpler warm-up probler
18

---

Published as a conference paper at ICLR 2026
first, then apply the pattern to the original numbers.
Focus on Constraints: Frame as puzzle with strict rules. FOV Shift: Narrate from the perspective of an element inside the problem.
Rules:
State your chosen Strategy first.
Core logic, numbers, and the final boxed answer must match
the ORIGINAL PROBLEM.
Avoia copying 5t consecutive words from the original text.
Response Format:
STRATEGY: [Chosen strategy]
REWRITTEN PROBLEM:
[Your transformed problem statement.
SOLUTION:
[Step-by-step solution matching the transformed problem. \boxed [Final answer])
ORIGINAL PROBLEM;
D.2 EXPERIMENT DETAILS
First, for all baselines, we use the official code provided their public repositories. For Co-Reward, models of the same size rewrite questions, instead of relying Qwen3-32B, ensure a fair comparison. When training the Qwen3-XB-BB version of Qwen3-XB for question rewriting. Similarly, for the Llama-Instruct series, use their respective instruct models. The prompt used for question rewriting follows the one provided in the original paper Zhang al. (2025).
series, we use the corresponding instruct
he following hyperparameters shared across all baseline methods and Self- Harmony. For the format coefficient and diversity coefficient, ususally we set as 0.1 for most cases. And few of experiment using other setting, check far setting these parameters.
Hyperparameters
EXTRA EXPERIMENTS
There are some extra studies in this section.
E CONVERGENCE OF PSEUDO-LABEL SELECTORS
At each training step, we use the identical rollouts from both views determine the pseudo- label chosen by two different methods: dual-view majority voting and our proposed Harmonic Mean Score (HMS) selector. Tigurepplor the difference Pass@) accuracy between these two selected labels.
The plot shows that HMS initially selects more accurate labels, providing crucial advantage when the model is uncertain. The difference between the two methods converges zero because our overall training dynamic successfully stabilizes the model's policy. As the model becomes more confident and its answer distribution sharpens, both majority voting and HMS will naturally select the same high-proc bighbbbbbbbiltty answer, causing their choices to converge. This convergence is evidence of successfully stabilized policy, initiated by the early robustness of the HMS criterion.
E.2 VALIDATING SEMANTIC CONSISTENCY OF THE REFRAMER
A critical requirement for Helf-Harmony is that the Reframer generates problems that semanti- cally equivalent to the originals. To validate this, we conducted an analysis using powerful, external Large Language Models (L.MSs) unbiased judges. We randomly sampled 200 problems and used
19

---

Published as a conference paper at ICLR 2026
Table Hyperparameters.
Hyperparameter
Value
Max prompt length
1024
Max response length
3072
Batch size
128 GSM8K
128 MATH500
80 AMC
24 AIME
MMLU-Pro
192 GPQA
Policy mini batch size
128 GSM8K
128 MATH500
80 AMC
24 AIME
32 MMLU-Pro
192 GPQA
Policy micro batch size GPU
8 GSM8K
MATH500
5 AMC
AIME
4 MMLU-Pro
GPQA
Learning rate
x 10 b
LR warmup steps ratio
0.1
hline Weight decay
0.01
Leaming rate warmup
cosine
Optimizer
Adam
Tempeeature
1.0 for Training
0.8 for Testing
Top k
-1
Top p
1 Training
0.95 Testing
Number of samples example n
16 GSMSK
16 MATH500
32 AMC
32 AIME
16 MMLU-Pro
16 GPQA
Number of samples per example for Testing
16
Remove padding
True
Use KL loss
True
KL loss coefficient
0.005
KL loss type
low var kl
Clip ratio
0.2
Grad clip
1.0
Temperature
1.0 Training
0.8 Testing
Verifier
MATH500 verl math verifier
GSM8K verl math verifier
AIME ttrl math verifier
AMC ttrl math verifier
GPQA url math verifier
MMLU-Pro ttrl math verifier
our trained Reframer to generate their rewritten counterparts. We then prompted two state-of-the-an models, GPT--40 and Claude 4 Sonnet, solve both the original and the rewritten versions.
20

---

Published as a conference paper at ICLR 2026
Pseudo Label Pass Difference
0.03
(HMS Majority Voting)
0.02
0.01
0.00
0.02
Step
Figure 9: Pass@1 accuracy difference between labels selected by HMS and dual-view majority voting, using the exact same rollouts at each step. The convergence towards zero indicates that as the model's policy stabilizes, both selection methods increasingly agree.
Table 4: Analysis of Reframer-generan problems using external oracle models(GSM8K)) The high consistency rates confirm that the rewritten problems are semantically equivalent to the originals.
Metric (Pass@) %)
GPT-4o
Claude 4 Sonnet
Original Questions
98.41
99.
Rewframed Questions
87.30
87.88
Consistency Rate
88.71
88.55
Our primary metric is the Consistency Rate, which measures the percentage of cases where the external oracle model produced the same final answer for both the original and rewritten versions of a problem. The results, presented Table show a high degree of consistency. For GPT-4o,, the consistency rate 88.71 %, and for Claude Sonnet,
88.55%.
This strong agreement demonstrates that our Reframer reliably generates new problems that preserve the core semantic meaning of the originals. This validation crucial, as it confirms that the second view used in our Harmonic Mean Score calculation is a valid and meaningful perspective on the original problem.
E.3 SENSITIVITY ANALYSIS ON SEMANTIC SHIFT
Smaller language models (0.6B) are more likely to introduce semantic drift, wording biases, or shal- low perturbations when paraphrasing, compared larger models (14B). To whether such im- perfections could cause the Harmonic Mean selector to incorrectly down-weight valid answers, we conduct stress test on the Confident-Corme Correct subset of the math dataset. This subset constructed by running Qwen3 models with 16-rollout majority voting and keeping only questions where the majority count exceeds half the rollouts and the majority answer is correct.
For each Solver (Qwen3-1.3 17BB-Bse. Owen3-4B-Bass. the Reframer size across 0.6B, 1.7B, and 14B parameters. This uses model scale as proxy for paraphrase quality: the 0.6B Reframer tends to produce fidee-fidelity, noisier paraphrases, while the 14B Reframer yields higher semantic fidelity. If Self-Harmony were highly sensitive to semantic shift, would expect performance to degrade when using small Reframers this already-correct subset.
(ive3-8B-Basse) we fix the Solver and vary
Instead, observe consistent and substantial gains over the baseline across all Solvers and all Re- framer sizes (Talle Even with the 0.6B Reframer, Halmooyr significantly improves Mean Pass @ 1 the Confident-Corret subset (like Qwen3-4B 4B-eess- 83.98 96.48%; Qwen3-8B-
21

---

Published as a conference paper at ICLR 2026
Table 5: Self-Harmony performance the Confident-Correet subset under different Reframer scales. For each Solver, we fix the Solver and vary the Reframer size; Baseline denotes initial model.
Self-Harmony / Confident-Corree Baseline
0.6B
1.7B
4B
14B
Qwen3-1.7B B.en-e
86.32
94.92
94.92
95.31
96.48
Qwen3-4B-Ban
83.98
96.48
97.65
98.04
98.36
Qwen3-8B-Base
89.06
97.85
97.75
98.02
99.07
Base: 89.06% - 97.85%). This demonstrates that the Harmonic Mean selector not overly sensitive small perturbations imperfect semantic equivalence: it continues to recognize and preserve valid answers despite noisy rephrasings. Performance further improves as Reframer size increases (up 99, 9,.07% with Qwen3-8B-Base + 14B Reframer), indicating that even the smallest 0.6B Reframer is already sufficient to support Self- Self-Harmony, and larger Reframers bring additional gains.
E.4
HARMONIC MEAN PSEUDO LABEL SELECTOR VS. HARMONIC MEAN REWARD
0,800
Math500 L3 Pass@11
0.750
0.725
0.700
0.675
Harmonic mean reward
Harmonic mean pseudo label selector
0.650
Step
Figure 10: Performance comparison on the Math500 Pass@ metric between the harmonic mean pseudo label selector and the harmonic mean reward baseline over training steps.
Since there are some paper only using internal signal rather than choosing pseudo label, an ablation study is conducted to compare the performance for harmonic mean in two different using way: 1. Pseudo label selector 2. Harmonic mean as reward. Experiment shows that pseudo label way has better performance as shown in Figure 10
E.5 ADDITIVE REWARD VS. MULTIPLICATIVE REWARD
To further investigate the effect of different reward aggregation strategies, we compare the per- formance of the additive reward and multiplicative reward schemes. The additive reward uses weighted sum of individual loss components, with weights set to match the proportional influence of each component in the multiplicative reward. Specifically, we aligned the weights based on the effective gradients derived from a first-order Taylor expansion. Our multiplicative reward is defined as:
Rmult = (1 - wy R1) - wa -a Ra) Ruaive For correct answers (Roolve and small penalty terms, this linearizes to:
Rmult R Ruave - wy Rg R.R
To control for hyperparameter influence, we configured the additive baseline (Rust Roove + wh R2 we R.) match this approximation strictly:wa 1w1 -W2 -Wdd This align-
22

---

Published as a conference paper at ICLR 2026
Math500 Pass@11
0.80
Additive Reward
0.76
Multiplicative Reward
60
80
100
Step
Figure 11: Comparison between the additive reward (blue) and multiplicative reward (orange) formulations on the Math500 L3 Pass@) metric. The additive reward combines components through weighted sum, while the multiplicative reward uses the product of the same components ratio. Both rewards are normalized to 1 for fair comparison, and all other training parameters are kept exactly the same.
ment ensures that both models are subject to identical penalty magnitudes for format and diversity violations. Consequently, the superior performance of Self- Helf-Harmony can be attributed to the struc- tural advantage of the multiplicative design (which acts as a strict "AND" gate) rather than dis- parate hyperparameter settings. For fair and controlled comparison, both rewards are normalized to 1, and all other hyperparameters and training settings are kept exactly the same. As shown in Figure I] while both methods achieve similar final performance, the multiplicative reward demon- strates slightly faster convergence in the early stages of training.
F EXAMPLES OF REWRITTEN PROBLEMS
To illustrate the role of the Reframer, we present examples of original and rewritten problems. Well-formed rewrites are shown in Table 6 while problematic cases are shown in Table 7 In the well-formed examples, the rewritten questions modify the narrative or context but remain function- ally equivalent to the originals, ensuring that the correct answers are preserved. In contrast, the problematic rewrites fail to meet these requirements, either by reducing to trivial restatements or by altering the correct answers, demonstrated in Table 7
G THEORETICAL DISCUSSION
G.1 PROOF OF THEOREM 3.2
Theorem G.1 (Harmonic Mean Selector from Invariant Infomax). Assume the view-invariance con- dition (Assumption 3.1 holds. Suppose moreover that the following conditions satisfied.
Non-degeneracy, For every label
pa(a) pi (a)) <
This simply excludes the trivial case in where the majority voting can solve.
Balanced- Confidece. There exists a constant E (0, 1) such that for every maximiser a" of Jx(.)
Ipo(a*) pi("a)) < x[po(a*) + +pi("") (aa)).
Then, for the penalty weight = 2, the pseudo label that maximises the recond-order approximation of the view-invariant Infomax objective J2(a) T(Za; A) (Za; X) is obtained by the harmonic
23

---

Published as a conference paper at ICLR 2026
Original Question
Rewritten Question
Let z be a complex number such that |=| 1. Find the maximum value of
Imagine you' re an elf tasked with finding the maximum healing power of two magical spells combined. One spell's strength is represented by [1 + z/. where x is a magical rune with known power level of I (æ = 1). The other spell's strength is 11=++2" = Your job figure out the maximum total healing power when using both spells together.
[] + -1++|+==+++
How many y-intercepts does the graph of the parabola x = y² -4 1 have? train has 172 people traveling on it. At the first stop 47 people get off and 13 more people get on, and at the next stop another 38 people get off. How many people are on the train?
A path is described by the equation æ = 5²² - 4y 1. How many points this path touch the vertical line at 0?
Imagine a train journey where the train starts with 172 people. The train first stops at station where 47 passengers disembark and 13 new passengers board the train. At the next station, 38 more passengers get off. How many people remain on the train as continues its journey?
A whirligig spins at five times the speed of a thingamabob. A whatchamacallit spins eleven times A
Imagine the relationship between these spinning objects as a program- ming task. The thingamabob is like a base class ('BaseObject') from which other objects inherit properties. A whirligig spins five times the speed of this base class, making a derived class ("FiveTimessO- ject'). Similarly, a whatchamacallit spins eleven times faster ("Fleven- TimesOhject", inheriting the speed from "BaseObject'. Given that the 'ElevenTTimesO Tinvennittebbeect" spins at 121 meters per second, determine the speed of the FiveTimesObjeet..
faster than thingamaboh. whatchamacallit spins at 121 meters per second. How fast docs a whirligig spin?
What fraction of fect is 3 inches? Ex- press your answer as a common frac- tion.
Imagine you'te 'r a craftsman dividing a 2- 2-foot-long wooden board into smaller, precise segments. You notice that you have a 3-inch piece that you need. What fraction of your 2-foot board does this 3-inch piece represent?
What fraction of 2 feet 3 inches? Ex- press your answer as a common frac- tion.
Imagine you are designing a digital game where objects need to be con- nected at specific distances. In the game, the main character moves by taking steps of 2 feet, but the tasks require the character to interact with objects placed every 3 inches. What fraction of the main character's s step length (2 feet) should be used to reach the nearest interaction point (3 inches)? Express your answer as a common fraction.
Let N be the greatest four-digit pos- itive integer with the property that whenever one of its digits changed to 1. the resulting number is divisible by 7. Let Q and Il be the quotient and remainder, respectively, when N is di- vided by 1000. Find Q + R. Find the value of such that
Find the greatest four-digit positive integer N such that changing any one of its digits
results in number that is divisible by 7. Then, determine the quotient Q and remainder It when N is divided by 1000, and compute Q + R.
Find a general formula solve the equation Gr 19r r, and then apply it to find the specific value of r.
4r for 2r-7
2 19r
2r 7 -.
Each vertex of regular octagon is in- dependently colored either red or blue with equal probability. The probabil- ity that the octagon can then be ro- tated so that all of the blue vertices end up at positions where there were orig- inally red vertices is m, where m and n are relatively prime positive integers.
Consider regular octagon with vertices labeled as 0, 2, ..., 7. We color each vertex either red or blue with equal probability (which we can think of as a coin flip). After painting, we can rotate the octagon (which we can think of as performing arithmetic modulo 8) to poten- tially reposition the blue vertices onto positions originally occupied by red vertices. What is the probability of this happening, expressed the form mn where m and n are coprime integers, and find m + m?
What is n??
Find the coordinates of point P the line y = -x + 6 such that the dis- tances from P to points A(10, - -00) and (O, 0) are equal. ie., PA = PO. Express your answer as ordered pair (a, b).
Consider a 2D plane where the line y =-x 6 is defined. Let's shift the domain to the set of complex numbers such that the imaginary part corresponds to the y-coordinate and the real part corresponds to the x- coordinate. We need to find complex number P' = a + bi on the line y x 6 (which becomes b -a + 6) that is equidistant from the complex numbers representing points A(10, -10) and O(0, 0). This means finding P such that the distances IP - A and IP - Ol are cqual. Express P as (a, b) in the original plane coordinates.
Table 6: Examples of original and well-formed rewritten problem statements. The rewrites are functionally equivalent to the originals, ensuring that the correct answer remains unchanged.
24

---

Published as a conference paper at ICLR 2026
Original Question
Rewritten Question
Find the coordinates of point / on the line = -x + 6 such that the dis- tances from P points A(10, -10) and (0, 0) equal, i.e., PA PO. Express your answer as an ordered pair (a,
Find the coordinates of point P on the line y -2 + such that the distances from P' to points A(10, -10) and (00,0) are equal, i.e. PA PO. Express your answer ordered pair (a, b).
Based these matrices, choose the correct statement. Options: A, Z and represent observables. B. (e*) ) Y
Imagine you re watching dramatic play called Q-Matrix. The differ- acts in this play are represented by matrices W, X, and Z. The director Z calls out, "W and You two are my favorite scenes. Any other order, pfft, they re iust smoke and mirror tricks!" What does the director mean by this?
(e (e-x) represents quantum state. C. W and represent the evolution op- crator some quantum system. D. There exists vector to which if one multiplies e the norm of the vector changes.
Let a, b, c, be positive real numbers. Find the minimum value of
Let S be a set of positive real numbers. Find the minimum value of S ( where S is a sum of n positive real numbers. Assume that S s n²
(a+b+c+dl (1+
1
Let a, b, e, positive real numbers. Find the minimum value of
Suppose that kids carry water from the kitchen to their bath in their houses. The four containers have capacities a, b, с and d litres, respectively. The kids have to carry enough water to fill their bath per + b d litres of water filled and cach kid takes turn carry the water. When carries water, uses 1/a, 1/b, 1/c and 1/d of time earry water per bath which equal to number of turns each gone the bath tub respectively. What is the minimum total number of turns which sum up to fill a b + d of water when each kid uses their own specific container? Moreover, during each time unit, the whole amount of water a. b, and liters filled drained from containers, such that the water level in each container remains the same. How many minutes does it take until every kid has filled their hath tub?
(a + b + (1+
Table 7: Examples of original and problematic rewritten problem statements. These rewrites either reduce to trivial restatements or alter the correct answer, violating functional equivalence.
mean of the two view-probabilities:
max arg (a)(a) (a) Po(a) + (i()
Proof of Theorem 3.2
First of all, let's define the two view and objective function:
The views are defined by the random variable X (,, sampled uniformly, that
p X = x)) = p(X = = !
For candidate answer a, write
Pu(a) = p(A = a X= = =x),
Pi(a) = p(A = a X x').
We shall also and the Bernoulii indicator Z, I(A a).
p(a) = 1/2 (po(a) + (a)),
8(a) = (po - -p(()),
Based on infomax principal, we introduce the invariant- Infomaxx score to be maximised:
Jx(a) (Z.: A) A >1 XX)
(3)
and the Theorem 3.2 sets = 2, where the objective function is
Jz(a) T(Za; A) (Z,; X)
25

---

Published as a conference paper at ICLR 2026
And now let's expand mutual information expression: with mutual information defined in two equivalent ways
I) -HU)) HU (W) Ew-w DRL(plU W =w) w) ((U))
Taking U = Za. W = A (+))
T T(Z:: A) = H(2)) HZa A).
Because Za a deterministic function of A (knowing A tells us with certainty whether = a), the conditional entropy vanishes: (Za A) 0. Consequently
I(Za; A) = (Za))
Marginally Za is Bernoulli with mean p(Z. = = p(AA = a) = p((a), so
H(Z.) = M)(aa), where h(x) = - In - -
- -1-x) In(1 - x))
the binary entropy.
Using the second form in (+) with U = Lu, W = X we obtain
[(ZuiX)= Σ p(x) DRu(p(Za x) (ll()))
(5)
Conditional laws. For X the answer distribution is Pa(a). For X x' the answer distribution is Pi (a); therefore Z, X = x ~ Bern(po(a)) (a)). Similarly Z = ~ (ern(p)(a)).
-Marginal law. By total probability and the uniform prior X, p(Z, 1) p/p(a + (a) p(a), so 7a ~ Bern (f(a)).
Substituting in equation
Denoting Bern(p) a Bernoulli distribution of mean p.
I I(Z,; X) (Bem((()))))) Bern(p(a))) + ( (a)) (era()
Let p(a) for brevity and define
s(p) [| = plnn P + (1 p) In
1 p
q
1
Taylor-expand around p = q:
1 1- p
g' (p) = In q 1 1-q" q¹
In
g"(p)
p +
1 1
so that g(9) = g'))) = and (g))) = 1//g(1 - -)). For = p - small,
2
0(q A)
2q(1 4) 0(A3).
Since Po = 5 and p1 = 8 we have g(po) (p1)) 8² 62/12q(1 - q)] +]] +083). Averaging the two contributions yields
8(0)
1(aa:)) (1 p(a))
+ +
(6)
So Second-order Taylor expansion approximation is:
5(a)2
J2(0) I(Z.: A) 21( h(p(a)) p(a) - (1-()
26

---

Published as a conference paper at ICLR 2026
Using Assumption 2 expand the denominator 1-p(a) 1 p(aa) + :(a)² +... and obtain:
8(a)2
8(a)²
(1 - ((a)) p(a) (pa))(((a) + + +....
R(a)
Balanced confidence (Assumption 2 implies (ō(a)) S k. (.)(),
x²p(a)2
[R(a)] - p(a) I 2x 2<²p(a) = O(p(a)),
which is third-order. Discarding R(a) gives the second-order approximation 5(a)²
Jz(a) ~ h(p(a)) p(a)
Next, by Assumption | we have p(a) for all a, and since the binary entropy h is strictly increasing on (0, $), maximising h(p(a)) is equivalent to maximising p(a) itself. Therefore
8(a)²
arg max J2(a) = arg max p(a) p(a)
Finally, using the identity
p(a) 8(a)2 pa) (a
p(a) po(a) + +Pi(a)
we define the harmonic-mean surrogate score:
S(a) 2 (a) (a) Pa(a) (a)
Thus, the pseudo-label selected by J2 satisfies
y² = arg max pa)(a) pi (a) Po(a) (a)
E arg max J2(a)
(8)
The proof only assumes two conditional answer distributions, (a p(a x= and (a
p(a X = x'), indexed by a binary view variable X. These views can be instantiated multiple
ways:
(i) two semantically equivalent queries, po(a) = p(a 1) and (a(a) = p(a x');
(ii the same query under different system prompts or, o', po(a) p(a o. and (1 p(a |
x, o');
any other mechanism that induces two correlated yet distinct answer distributions. Because the derivation treats abstractly, the harmonic-men mean selector remains valid for all such pairings.
G.2 GENERALIZATION OF THEOREM 1: TUNABLE À AND GENERALIZED MEANS
In this part, we extend the derivation of the Invariant Infomax objective to the general case where the penalty weight tunable. We demonstrate that varying À is theoretically equivalent selecting a specific Generalized Mean for aggregating binary view probabilities. Recall the Taylor expan- sion of the Invariant Infomax objective (a) I(Z:: A) XI (Z:: Using the second-order approximations derived in the main proof, the objective simplifies to:
5(a)2
JA(a) = p(a) 2p(a))
where p(a) = po(a)+pp(o) is the mean probability and 6(a) = po(a)-pi(a) is the semi-difference
2
between views. Note that 6(a) is exactly equivalent to the statistical variance the two view probabilities.
27

---

Published as a conference paper at ICLR 2026
tpl tp
1/R
Now, consider the Generalized Mean with exponent defined as Ms (po, pi) = second-order Taylor expansion of Mj around the mean p is given by:
The
8(a)2
(& P1) p(a) (h: 1) 2p(a)
(10)
By equating the coefficients of the variance term 8(a)² in Eq. equation 9 and Eq. equation 10 establish a direct mapping between the penalty weight À and the mean exponent
- = k-
= =
(11)
This mapping allows us to categorize the behavior of the selector across the spectrum of X. In the list below, we describe the penalty regarding the variance term 8(a)2:
A - = h = 2 (Quadratic Mean RMS): The objective effectively adds a variance bonus (z + 22 This is "risk-seeking" and prefers disagreement, making it unsuitable for consistency checking.
8²
À = k 1 (Arithmetic Mean):
The objective maximizes p (Standard Voting). It ignores view consistency entirely (coefficient 8² 0).
À =1 k 0 (Geometric Mean): The objective maximizes /PoPi. It penalizes
variance with a coefficient of -1 (r= p
2p
A 2 k -1 (Harmonic Mean): The objective maximizes 2poPt, It penalizes
potpi
variance with a coefficient of -2 (~= p 2 2 to the Geometric Mean.
effectively doubling the penalty compared
G.3 MULTIPLE VIEWS CASE AND GENERALIZED HARMONIC MEAN
In this part, we extend the analysis of the Harmonic Mean Selector from the binary view setting (K = 2) to the general multi-view setting (K 2 2). We show that maximizing the view-invarian theoretically equivalent to maximizing the Generalized Harmonic
Infomax objective with Mean of the view probabilities.
Theorem G.2 (Generalized Harmonic Mean Selector). Let there be K views defined by ran- dom variable uniformly distributed over (I1,...,, T.. Let Pa(a) p(A a X sx) be the prediction probability for candidate under view k. Under the same assumptions of non- degeneracy and balanced confidence as Theorem 3.2 maximizing the second-order approximation of the objective Jz(a) I(Zo; A) (Z; X) recovers the Generalized Harmonic Mean of the K probabilities:
K
y = arg max
K
(a(aa
Proof. We analyze the objective function and the harmonic mean separately using Taylor expansions around the arithmetic mean. Let the arithmetic mean be p(a) * Σk (a((a and the variance across views be (² 1/4 ( (px p(a))2
1. Expansion of the Infomax Objective. The mutual information / [(a,;A) A) remains the entropy of the marginal Ze, which approximates to the mean probability for small p:
T(Z.; A) ()(a)) es p(a) (up to first order).
The mutual information between the pseudo-label and the views, I(Z,; X), is the average KL- divergence between the specific view distributions and the mean distribution:
K
1
I(Z,; = K
DEL (px))) || Bern(p(a)))
k=
28

---

Published as a conference paper at ICLR 2026
(p-9)² 2q(1-9)
Using the second-order Taylor expansion of the Kl-ddiveg,ence DRL.P 11
we obtain:
1
K
(px - -(a)))
)²(a)
T( X) ~
K
k=1 (1
21(a) p(a))
Applying the simplification from the proof Theorem 3.2 dominated by the first order behavior), the objective with penalty X = 2 becomes:
(where the denominator term - -p(a)
(a)
=
(12)
p(a)
Expansion of the Generalized Harmonic Mean. Let H K (a) denote the harmonic mean of proba- bilities (pz (p) (a)). We approximate terms of the form 1 (/ps(a) using Taylor expansion of f(x) = 1/2 around the mean p(a):
1
(pk
(pk 2)
Pk
p2
Averaging these inverse terms K views:
K
02
E(p
p)
K
Kp
p3
p
Finally, we invert this sum to get (x(a). Using the approximation (A+ 2 A A=
-1
02 = p(a) 0²(a) R - T
Hx(aa =
(13)
D
F(a))
Comparing Eq. equation 12 and Eq. equation we see that maximizing the Infomax objective J2 equivalent (at the second order) maximizing the Generalized Harmonic Mean. The variance aa( acts as a penalty, discouraging answers where the model confidence fluctuates heavily term p(a) across views.
G.4 NON-UNIFORM PRIOR AND WEIGHTED INVARIANT SCORE
Now we generalize Theorem 3.2 to the case of a non- non-uniform prior over views. Let the prior over the views be parameterized by π E 10, such that:
p(X- x) I
p(X x( =- n
The marginal probability of answer becomes the weighted average:
(a) (Po(a) + (1 m))p (a).
The definition of the Mutual Information J(Za; X) generalizes to the weighted sum of KL diver- gences:
T(Z,; X) n DKL (po(a) (.) + D) (p1 (a) || (a)).
(14)
Recall the second-order approximation of the KL divergence (KL |l q) = (p-9)² the deviations of each view from the weighted marginal. Let A(a) = po(a) PI (a). Then:
20(1-q1' We compute
Po(a) p- (o(a) (xpo(a) + (1 (a))
=(1 - -T-)((po(a) - (a)) (1 ((a).
(i(a) (a(a) = (a) - [apu (a) + (1-))p((a))
=-m(po(a) p1()) = AA)
29

---

Published as a conference paper at ICLR 2026
Table 8: Comparison between generalized mean and TTRL baseline in fine-grained dataset, Self- Harmony achieves best performance
Method Conf. Correct
Conf. Incorrect Not-Conf. Correct
Not-Conf. Incorrect
Arithmetic 98.83
76.95
90.82
58.13
Harmonic
99.61
77.73
93.28
61.25
Quadratic
98.43
76.25
89.61
60.98
Geometry
99.22
76.17
91.23
58.89
Full Infomax
99.22
75.78
90.10
60.00
TTRL
98.90
76.81
92.35
58.97
Substituting these into the Taylor expansion of Eq. equation
14
1
X)
π A()) (1- (1-)))
2pr (a)(()( (m))
A(a)
[ x)² + 72 x]
2p, (a)(1 Pn (a))
((1-m)(1-xt) (1==))()-
x(1 ())a)
2px (a)( (a))
Assuming the The objective function (2(a) then becomes:
condition holds, we approximate the denominator 1 (x)
Jz(a) (a) 7(1 - (p) (i))
(a(aa
This Weighted Invariant Score generalizes the Harmonic Mean Selector. When π 0.5, the variance term T11 x) 0.25, recovering the original form. For π * 0.5, the objective balances the weighted confidence against the consistency penalty, where the penalty strength is modulated by the prior variance.
H EMPIRICAL RESULT FOR GENERALIZED MEAN COMPARISON
We further conducted fine-grained analysis using Qwen-4B-Bas on the Math dataset shown in Ta- ble We stratified the problems into four categories based on the base model's inherent confidence and correctness:
Confident (majority answer 2 N/2 votes) and correct
Not confident (majority answer N/2 votes) but correct
Confident and incorrect
Not confident and incorrect
Across all four subsets, the harmonic mean yields the best performance, including the most chal- lenging confident and incorrect region where models typically hallucinate with high certainty. These results demonstrate that Self- Harm-Hy improves mean pass@) I 16 rollouts not only in easy high- confidence settings but also under uncertainty and erroneous certainty, highlighting its robustness and general applicability.
INFERENCE-ONLY COMPARISON
To evaluate the harmonic mean pseudo-label selector without any training, compare Majority Voting versus Self-Harmony under different inference budgets. Following the Confident Correct definition in Section H sweep the number of rollouts from 32 to 256 and report Pass (in %) based on the selected answer among the rollouts shown in Table g
Confident-Correct Dataset
30

---

Published as a conference paper at ICLR 2026
Table 9: Inference-only performance (Pass @ %) under different rollout budgets. Self-Harmony consistently improves or maintains performance compute increases, whereas Majority Voting may plateau or degrade.
Subset
Model
Method
32
64
128
256
Confident-Corres
Qwen-4B- Base
Majority Voting
99.61
98.83
98.44
98.44
Confident-
Owen-4B-Ban
Self-Harmony
99.61
99.61
99.61
99.22
Confident Confident-Correct
Qwen-8B 8B-Base
Majority Voting
98.83
99.22
99.22
99.22
Conident-Correst
Qwen- 88-88B--Bs
Helf-Harmony
98.83
99.61
99.61
99.61
Confident-Incorree
Qwen-4B-Base
Majority Voting
75.78
76.95
75.78
75.39
Confident-Incorme
Qwen-4B-BB
Self-Harmony
78.52
78.12
77.34
77.73
Confident-Incorrect
Owen-8B-Base
Majority Voting
70.70
70.31
69 ,29
69.14
Confident-Incorreet
Qwen-8B-Bas
Self-Harmony
71.87
71.48
70.70
71.48
Not-Confident-Comet
Qwen-4B -ass
Dwen-4B-Base
Majority Voting
90.23
90.23
91.40
91.40
Not-Confident-CC
Qwen-4B-Baan
Self- Heff-Harmony
95.31
94.92
94.53
94.53
Not-Confident-Corme
Qwen-8B-Bas
Majority Voting
89.45
89.84
90.62
91.01
Not-Confident-Comeee
Qwen-8B-Base
Self- Herf-Hyrm
94.14
94.92
94.92
95.31
Do-Conffident-nnooreed
Qwen-4B-BBa
Majority Voting
64.45
64.84
66.01
65.23
I--onffieett-Incoreett
Qwen-4B-Base
Self-Harmony
65.62
66.41
66,80
67.19
Nol-Confident-Incorreee
Qwen-8B-Base
Majority Voting
60.93
63,28
63.67
63.28
Aol-Confident-nncorreett
Qwen-8B-Base
Self-Harmony
61.32
64.06
65.23
65.23
AMC
Qwen-4B-Bass
Majority Voting
54.22
53.01
54.22
54.22
AMC
Qwen-4B-Bas
Self- Har--Hyr
56.63
56.63
57.83
57.83
AMC
Qwen-8B-N -an--8B-Basee
Majority Voting
54.22
56.63
55.42
56.63
AMC
Qwen-8B- Owee-8B-Basee
Self- Self-Harmony
57.83
59.04
59.04
59.04
Qwen-4B-Base: Majority Voting slightly decreases rollouts increase (99.61% 61%
98,44%) whereas Self-Harmony remains highly stable at 2 99.22% across all rollout bud- gets.
Qwen-8B-Base Both methods improve with more rollouts, but Self-Harmony consistently converges to the highest accuracy (99.61 1%) and maintains from to 256 rollouts, while Majority Voting plateaus at 99.22%.
Confident-Incorreet Dataset
Self-Harmony outperforms Majority Voting at every rollout setting and remains stable as rollouts increase; Majority Voting fluctuates and never surpasses Self- Harmony.
Qwen-4B-Base:
Qwen-8B-Base:
Self-Harmony consistently outperforms Majority Voting across all rollout budgets, while Majority Voting steadily decreases as the number of rollouts increases.
Not-Confident-Correet Dataset
Qwen-4B-Base:
Majority Voting improves only slightly with more rollouts (90.23% delivers large and consistent gain (+4-5%) and remains
91.40%) whereas highly stable across rollout budgets (95.31% 94.53%). demonstrating strong correction ability when initial model confidence is low but the answer is correct.
Qwen-8B-Bane Majority Voting improves with additional rollouts but remains far below Self-Harmony every setting (89.45-91.00
94. 14.144955311% Self-Harmony quickly saturates at high accuracy and preserves it across budgets, indicating that the method reli- ably preserves valid low-confidence answers without being swayed by sample variance.
Mot-ConfiienttInnorreed Dataset
Qwen-4B-Basse Self- Helf-Harmnny consistently improves performance as rollouts increase (65.62% 67.19%), whereas Majority Voting remains stagnant and even decreases
31

---

Published as a conference paper at ICLR 2026
high rollout counts (64.45% 65.23%). This shows that Self- Self-Harmony helps correct low- confidence mistakes rather than reinforeing them.
Qwen-8B-Basee Majority Voting shows modest movement with additional rollouts (60.93-63.67%) but never closes the gap with Self-Harmony. In contrast, Self- Harmony scales positively with budget and converges to a much higher accuracy range (64.06-65.23)) 2,.2%),... indicating that the harmonic selection mechanism is robust against noise when confidence low and the answer is wrong.
AMC
Owen-4B-Base Self-Harmony outperforms Majority Voting at all rollout budgets and con- tinues to improve as the sampling budget increases, whereas Majority Voting remains un- changed or fluctuates.
Qwen-8B-Base Base: Majority Voting improves only modestly with additional rollouts, while Self- Half-Harmnyy achieves the highest accuracy and sustains from 64 256 rollouts.
Overall. Self-Harmony scales favorably with larger models and can be used in plug-and-play manner at test time to obtain further accuracy gains without any parameter updates or training.
COMPARATIVE SIGNAL QUALITY ACROSS CONFIDENCE CONFDENCE-CORRECTTEES CORRECTNESS QUADRANTS
To pinpoint the source of Self- Seff-Harmony's performance gains, we conduct fine ene-grained analysis of the training signals. categorize the base model's initial outputs into four subsets based on con- fidence and correctness: (1) Confident Correct, (2) Confident Incorrect, (3) Not-Confident Correct, and (4) Not-Confident Incorrect.
As summarized in Table 10 consistent pattern emerges across all evaluated metrics Spilissanpp correlation, score, Average Pass@ I, and pseudo-label accuracy:
Confident Correct: While both approaches perform well due to saturated correctness
in this subset, Self-Harmony demonstrates greater stability (like 99.61% % vs. 98.90% in Pass@ I), effectively preventing performance degradation.
Confident Incorrect: This represents the most challenging subset. Self-Harmony achieves substantial gains here (like +21.3% in Spearman p). indicating superior capability to suppress confidently wrong answers that typically mislead Majority Voting.
Not-Confident Correct: Self- Helf-Harnnny recovers weak-but-correct signals more effec- tively, yielding cleaner training supervision for ambiguous or borderline questions.
Not-Confident Incorrect: Even when neither confidence nor correctness is reliable, Self-
Harmony maintains discriminative supervision compared to the baseline.
Collectively, these results demonstrate that Self-Harmony's improvement is derived merely from "easy" samples. Instead, the framework significantly enhances pseudo-label quality in the critical areas where Majority Voting is fragile: specifically, confidently wrong and ambiguous instances. This robustness explains why downstream learning benefits more from Self- Helf-Harmoy than from standard self self-consistency aggregation.
K INTERPRETATION OF HARMONIC MEAN
Harmonic Mean 2po(a)pz(a Po(a) + (a)
(Po(a) (1 (a (a
(po(a) pa)a
Self Consistency
Invariance
This decomposition provides clear interpretation of the harmonic mean pseudo-labeling objective. The first term, highlighted green, rewards high overall confidence from both views analogous to maximizing the likelihood under view-agnostic aggregation. The second term, highlighted in
32

---

Published as a conference paper at ICLR 2026
Table 10: Performance breakdown across four confidence-corrretness quadrants. Self-Harmony consistently outperforms Majority Voting (MV) in the most difficult subsets (Confident Incorrect and Not-Confident Correct) across all metrics.
Confident Correct
Confident Incorrect
Not-Coufident Correct
Not-Confident Incorrect
Metric
Method
Spearman Correlation
Self- Harmony
18.35
36.51
37.16
30.11
(p, %)
Majority Voting
35.82
15.19
35.83
29.34
FI Score
Sclf- Har--Hyr
99.35
75.76
93.07
66.43
(%)
Majority Voting
88.89
69.87
91.26
63.41
Pseudo-label
Self-Harmony 99.22
79.69
95.31
68.75
Accuracy (%)
Majority Voting
96.09
65.62
90.62
60.16
Avg. Pass
Self Harmony
99.61
77.73
93.28
61.25
(16 rollouts, %)
Majority Voting
98.90
76.81
92.35
58.97
0.50
(90
(a) Pseudo-Label Accuracy
(b) Pseudo-Label F1
(c) Pseudo-Label Spearman
Figure 12:
Comparison of pseudo-label accuracy, Fl, and Smoothed spearman correlation across training steps for TTRL, Co-Reward, and Self-Harmony on Math500 dataset. Self-Harmony pro- vides consistently stronger and more stable training signals than the baselines.
blue, penalizes disagreement between views, acting view-invariance regularizer. Thus, the harmonic mean simultancously prefers labels that are (i) confident according to both views and (ii) consistent across views, rather than disproportionately influenced by single overconfident but unreliable prediction.
L COMPARISON OF TRAINING SIGNALS
Figure 12 compares three key training-signal properties-ppeudoollne accuracy. F1, and smoothed Spearman correlation-acct across optimization steps for TTRL, Co-Reward, and Self-Harmony on Math500 dataset. Across all metrics, Self-Harmony consistently provides the strongest and most stable supervision.
M USAGE OF LARGE LANGUAGE MODELS
In preparing this paper, used Large Language Models only aid with language polishing, gram- mar refinement, and improving readability. All research ideas, methodological contributions, ex- perimental design, and analysis were conceived and carried out entirely by the authors. The LLM was not involved in ideation, technical writing of results, or scientific claims. The authors take full responsibility for all content of this paper.
33


Reviews:
[{"review_id": "xaExdaPGI0", "reviewer": "Reviewer_v4TH", "summary": "The authors propose **Self-Harmony**, a new TTRL framework inspired by the Infomax principle. The core intuition is that the correct answer should appear consistently (robustly) across different versions of the question. The authors demonstrate that the argmax of the harmonic mean of the two probabilities under two different versions of the question approximately yields a good pseudo-label, and this is algorithmically achieved via appropriate prompt-engineered self-play. Its empirical efficacy is shown across various reasoning benchmarks and Qwen & Llama models, showing its superiority over prior TTRL approaches.", "questions": "1. Is there any connection between the core intuition here and group-invariant learning [1] or invariant risk minimization [2]? I don't recall any discussions relating to this line of works in the paper.\n\n2. (Minor) Are there any literature from human psychology or education for the claim \"common human robustness heuristic: when confronted with uncertainty, people often check solutions ...\" (line 161)?\n\n3. What happens if $\\lambda$ is also tunable? In other words, under two views, would one still be able to derive Eqn. (1) as a function of $\\lambda$? If that is the case, then what is the impact of tuning $\\lambda$ across various values?\n\n4. Similarly, with fixed $\\lambda$, what happens if one considers more than two views? For instance, when $\\lambda = 2$, then does multiple views lead to the general form of the harmonic mean? What might be the impact of having more views? At least from theoretical viewpoint, more (diverse) views should give better and better performance, although there is a possibility of plateauing over certain threshold.\n\n5. Follow-up question: If it is possible, I wonder what the impact of the second-order approximation is. Yes, computationally, having a closed-form solution is hugely beneficial, but the second-order approximation of the KL is tight only when the two probabilities are quite similar. Thus, if one puts in enough budget, I wonder if maximizing the infomax objective directly (with an appropriate KL estimator) gives better pseudo-label, and thus better performance. Or, whether the performance gain is minimal and the approximation objective is good as it is.\n\n6. Also a follow-up: the authors consider uniform prior over the views. I wonder if one could be a bit more clever here, e.g., depending on the \"persona\" or the \"strengths\" of the current LLM, one could take that into account to make the prior non-uniform?\n\n7. (Minor) As in many infomax papers (and as a theoretician myself), I wonder if this formulation of infomax (and its approximation) has been theoretically studied before, i.e., whether prior generalization bounds are applicable here (e.g., [3]). Or does this warrant further investigation for future work? If yes, which part of the new formulation yields novelty compared to prior literature?\n\n8. (Minor) Very recently, there have been some issues regarding floating-point operations (https://x.com/QPHutu/status/1984258808332550245). Can the authors comment on whether this issue(?) applies to their experiments?\n\n\n\n[1] https://www.jmlr.org/papers/v21/20-163.html\n\n[2] https://arxiv.org/abs/1907.02893\n\n[3] https://www.sciencedirect.com/science/article/abs/pii/S1566253525008383 (arXiv: https://arxiv.org/abs/2501.16768)", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Well-written\n- Clear and solid intuition\n- Theoretically well-grounded\n- Extensive experiments and benchmarks, as well as publicly released anonymized codes. Ablations provide a complete understanding of each component of the proposed algorithm and clearly show its superiority.", "weaknesses": "- The authors mention that the reward design plays a role in obtaninig SOTA performance (Section 4.3), but the only mention of how the weights involved with the rewards are set ($w_f, w_d$) is in Appendix E.4, mentioned in the passing only \"weights set to match the proportional influence ...\". This design choice should sufficiently be elaborated as well.\n- Although there isn't anything (significantly) wrong with the theoretical discussions, I believe there are some parts that warrant further discussion:\n  - I think it is beneficial to explicitly state the \"second-order approximation\" in the main text for the camera-ready version.\n  - The assumption that the view random variable $V$ is uniform is implicit and made explicitly only in the proof. This should be made clear in the main text.\n  - The theory is only derived for two views and $\\lambda = 2$ and uniform prior. I believe that more discussions on this would make the paper better; see Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hdDU0yyiBP", "reviewer": "Reviewer_dk94", "summary": "This paper proposes Self-Harmony, a label-free test-time reinforcement learning (TTRL) framework that improves reasoning stability without external supervision. The key idea is that a correct answer should remain consistent across both an original question and its paraphrase. A single model plays two cooperative roles: a Solver that answers and a Reframer that rephrases the question to provide a second view. Pseudo-labels are then chosen using the harmonic mean of answer frequencies from both views, emphasizing solutions robust to rewording while down-weighting view-specific errors. The authors provide theoretical justification for this criterion and integrate it into a self-play reinforcement loop. Across five reasoning benchmarks (MATH500, GSM8K, AIME24, GPQA, MMLU-Pro) and five open-source models, Self-Harmony achieves new state-of-the-art results—ranking first in 28 of 30 settings—and demonstrates superior training stability and pseudo-label accuracy.", "questions": "Could you provide more intuition or empirical evidence for why the harmonic mean is superior to arithmetic or geometric averaging in this setting?\n\nHow sensitive is the performance to small perturbations in the paraphrased questions or to imperfect semantic equivalence?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper stands out for its clarity and strong empirical results. The idea of enforcing consistency across paraphrased versions of a question is simple yet powerful—it directly targets a core weakness of existing test-time RL methods that overfit to their own biases. Using a single model in dual roles (Solver and Reframer) is elegant and avoids the need for extra models or supervision, making the approach practical and scalable. The harmonic mean pseudo-label rule is theoretically grounded and nicely bridges information-theoretic reasoning with an intuitive motivation. \n\nThe experiments are extensive and convincing: strong gains across five benchmarks, stable training curves, and robustness across model sizes all strengthen the paper’s credibility. Overall, Self-Harmony feels like a solid, well-executed contribution that advances test-time adaptation in a meaningful and reproducible way.", "weaknesses": "The harmonic mean rule, while supported by a theoretical derivation, still feels somewhat heuristic in how it’s applied. In practice, the paper does not deeply analyze why the harmonic mean performs better than simpler alternatives like averaging or weighted voting, or under what conditions it might fail. For instance, if the paraphrasing step inadvertently shifts the semantics of the question—introducing small wording biases or contextual cues—the harmonic mean could incorrectly down-weight valid answers. A sensitivity analysis or qualitative study of such “semantic drift” cases would make the claim of robustness more convincing.\n\nFinally, the “self-contained” assumption might break for smaller or less instruction-tuned models, since paraphrasing quality heavily depends on the model’s own linguistic competence—an issue not fully explored in the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dsY2YbU2Ze", "reviewer": "Reviewer_962B", "summary": "This paper introduces Self-Harmony, a framework where a single model plays two roles: a Solver that answers questions and a Reframer that rephrases them. The method aggregates answer frequencies across the original and reframed questions using the harmonic mean, which is shown to outperform baselines such as majority voting on the original questions.", "questions": "As mentioned in Weakness 1, I’m curious to see a breakdown showing where the improvements primarily originate compared to methods like majority voting.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Soundness**: The proposed method is intuitive and straightforward to implement. It builds on a sound intuition — that a correct answer should remain consistent across semantically equivalent but stylistically distinct formulations of the same question. The paper also provides a comparative analysis of pseudo-label quality between Self-Harmony, TTRL, and Co-Reward, demonstrating that Self-Harmony produces the highest-quality pseudo labels.\n\n**Substance**: The experiments are comprehensive, covering multiple model families including Qwen3 and Llama 3.1/3.2. The proposed approach shows consistent performance improvements across these models. Moreover, the ablation studies indicate that the training process is robust to different hyperparameter settings.", "weaknesses": "**Missing analysis**: It would strengthen the paper to include a deeper analysis of why Self-Harmony improves pseudo-label quality. For instance, one could perform a simple inference-based categorization: for each question, sample multiple responses and classify questions into four types based on model confidence and correctness:\n\n- Confident (majority answer ≥ N/2 votes) and correct\n\n- Not confident (majority answer < N/2 votes) but correct\n\n- Confident and incorrect\n\n- Not confident and incorrect\n\nThen, apply Self-Harmony method for generating pseudo-label to analyze where the improvements primarily arise — e.g., whether it mainly helps on low-confidence or high-confidence cases, or corrects certain systematic errors.\n\n**Minor issue**: In Table 2, the caption should appear above the table.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ixc08mC7w4", "reviewer": "Reviewer_RRhh", "summary": "This paper presents \"self-harmony\" an alternative to final-response consistency like self-consistency often used for test time scaling and more importantly for unsupervised supervision at test time. While self-consistency involves temperature sampling multiple solutions and selecting the one corresponding to the majority answer, the authors propose looking at the same problem from \"multiple views\" done by creating multiple rephrasings of the input query and selecting the answer consistent across these views by using the harmonic mean. The model is then trained on this unsupervised signal and the performance is reported. The authors argue that self-harmony is useful even in cases when self-consistency fails, i.e., model consistently arriving at the wrong answer.", "questions": "In addition to weaknesses, I have the following questions about the experimental setup:\n- What is the size of the training dataset, are we to assume that the test set (without labels) was used as the unsupervised signal? Is the same model trained for multiple tasks or different ones, please specify and compare against the other case? If the training is done on test queries (which are rather small -- at least for Math), how does the performance scale as the amount of training data increases?\n- Why do the authors use base model for Qwen but instruct models for Lllama?\n- Why not have GRPO with a trained reward model (since the test set is mainstream evaluation for reasoning tasks) aimed at reasoning as a baseline too and show self-harmony outperforms it?\n- Regarding 3(a), how often does the collapse happen and is it just on this specific dataset and/or model type? What are possible explanations or is it the entropy collapse seen in RLHF and how does the step where it happens relate to the size of the training set and the batch size?\n- Previous works show that self-consistency is useful even as the model size scales https://arxiv.org/abs/2509.06870, does the same hold for self-harmony. Can self-harmony be used without training to get more gains on the testsets?\n- The main paper should also contain examples of the rephrasings and any evaluation done to ensure the content and original intent were preserved.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Obtaining unsupervised signals for improving reasoning performance is an important area of work with potential applications in scaling reasoning performance during pretraining or in the wild, where annotations are not available. This paper makes a contribution towards the broader goal.\n- The paper is well-structured and easy-to-follow with decent empirical gains in the main resutls as well as some theoretical justifications for the view-invariant info-max objective.", "weaknesses": "- Missing connections to previous work in \"unsupervised\" training signals for reasoning: \n    - https://arxiv.org/abs/2411.04109\n    - https://arxiv.org/abs/2210.11610\n    - other papers on RL with external reward models for reasoning: https://arxiv.org/abs/2406.16838, https://arxiv.org/abs/2403.13787, etc.\n- While the main motivation hinges on self-consistency being ineffective when the model is consistently wrong, the citations and experiments to back this up are lacking:\n    - To the best of my understanding, Liu et al 2025b uses self-consistency to compare across different prompting strategies and the shortcoming of self-consistency at large was not clear to me, the authors need to be more explicit when making this connection (L157-160).\n    - Since the work requires actual model training on this unsupervised signal, simply the accuracy of the pseudo label does not suffice (as in Fig 3b). Unclear if the original test set (ground truth labels) is biased towards getting correctness right, or are we measuring the how often the consistency indicator lines up with correct or incorrect prediction. It would be prudent to report F1, or correlation between the metrics and the binary ground truth by Spearman or Somer's D correlation as done in https://arxiv.org/abs/2411.04109.\n    - As shown in previous works, https://arxiv.org/abs/2411.04109 and https://arxiv.org/abs/2509.06870 (concurrent), as the size of the majority answer increases, self-consistency becomes a better correctness indicator. Therefore, the authors should show that self-harmony outperforms self-consistency when setting consistency threshold to filter out smaller answer sizes. \n- Also see clarifications on experimental setup underspecified in the paper below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)