Semantic Voting: A Self-Evaluation-Free Approach for Efficient LLM Self-Improvement on Unverifiable Open-ended Tasks
Authors: Chunyang Jiang
Yonggang Zhang
Yiyang Cai
Chi-Min Chan
Yulong Liu
Mingming Chen
Wei Xue
Yike Guo
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
unsupervised learning
self-improvement
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: The rising cost of acquiring supervised data has driven significant interest in self-improvement for large language models (LLMs). Straightforward unsupervised signals like majority voting have proven effective in generating pseudo-labels for verifiable tasks, while their applicability to unverifiable tasks (e.g., translation) is limited by the open-ended character of responses. As a result, self-evaluation mechanisms (e.g., self-judging and entropy minimization) are predominantly used to derive pseudo-labels. However, self-evaluation relying on LLMs typically incurs high computational overhead and introduces overconfidence issues due to intrinsic biases. To address these challenges, we propose a novel self-evaluation-free approach for unverifiable tasks, designed for lightweight yet effective self-improvement. Inspired by majority voting commonly employed in verifiable tasks, we propose semantic voting as a novel mechanism that relaxes the principle of hard matching (i.e., exact matching) toward soft matching (i.e., semantic similarity). Soft matching is achieved by leveraging a lightweight sentence embedding model to quantify semantic similarity, thereby mitigating excessive computational burden and intrinsic bias-associated limitations of self-evaluation. Comprehensive experiments demonstrate that our method achieves substantial gains in computational efficiency and overall better performance than self-evaluation methods across diverse model architectures and tasks.

Published as a conference paper at ICLR 2026
SEMANTIC VOTING: A SELF-EVALUATION-FRER APPROACH FOR EFFICIENT LLM SELF-IMPROVEMENT ON UNVERIFIABLE OPEN-ENDED TASKS
Chunyang Jiang, Yonggang Zhang, Yiyang Cai, Chi-Min Chan, Yulong Liu, Mingming Chen", Wei Xue, and Yike Guo Hong Kong University of Science and Technology cjaingag@connect. ust.hk
ABSTRACT
The rising cost of acquiring supervised data has driven significant interest in self- improvement for large language models (LLMs). Straightforward unsupervised signals like majority voting have proven effective in generating pseudo-labels for verifiable tasks, while their applicability to unverifiable tasks (e.g.. translation) is limited by the open-ended character of responses. As result, self-evaluation mechanisms (e.g., self-judging and entropy minimization) are predominantly used derive pseudo-labels. However, self-evaluation relying LLMs typically in- curs high computational overhead and introduces overconfidence issues due to intrinsic biases. To address these challenges, we propose novel self-evaluation- free approach for unverifiable tasks, designed for lightweight yet effective self- improvement. Inspired by majority voting commonly employed in verifiable tasks, we propose semantic voting as novel mechanism that relaxes the principle of hard matching (i.e., exact matching) toward soft matching (i.e., semantic simi- larity). Soft matching is achieved by leveraging a lightweight sentence embedding model to quantify semantic similarity, thereby mitigating excessive computational burden and intrinsic bias-associatee limitations of self-evaluation. Comprehen- sive experiments demonstrate that our method achieves substantial gains in com- putational efficiency and overall better performance than self-evaluation meth- ods across diverse model architectures and tasks. We have released our codes at http://hhttbb..oooooooocccccoooo
INTRODUCTION
Recent advances in large language model (LLM) research and engineering (DeepSeek-Al et al.
Comanici et al. 2025 Yang et al. 2025) have led to a surge in demand for high-cost su- pervision signals, such as expert-annotated reasoning trajectories (Xia al. 2025). fine-grained human feedback (Wu et al., 2023), and deliberately designed verifiers (Chervonyi et al. 2025). These supervision methods typically require substantial human effort and often suffer from limited generalizability. resulting in high data acquisition costs (Villalobos et al. 2024). To address these challenges, L.LM self- seff-improvement (Huang al.) 2023) has emerged as a promising paradigm, en- abling self-training through unsupervised (Zuo et al. (025) or self-supervised (Yuan et al., (024) generation of pseudo-labels, thereby breaking free from reliance on external costly supervision.
2025
One of the most prominent self-improvement approaches is building pseudo-labels via majority voting (Wang et al. 2003), which generates multiple response candidates for a given question and selects the most frequently occurring answer as the pseudo-label. This mechanism has proven to be a simple yet effective strategy across diverse training paradigms (Huang et al., 2023 Prasad et al, 2024 Zuo al. 2025). However, majority voting relies exact matching, limiting its applicability to closed-ended tasks, such as arithmetic problems and multiple-choice questions.
To realize gel--improvement unverifiable open-ended tasks, two self-evaluation-baie based strategies are predominantly adopted. The first, self-judging, leverages "LLM-as-a- Judge" paradigm, where
The corresponding authors are Mingming Chen and Yike Guo.

---

Published as a conference paper at ICLR 2026
the model evaluates its own generated responses by assigning scores or preferences, enabling self- rewarding without reliance on external feedback (Yuan et al. 2024). The second is based on entropy minimization, which estimates the confidence of generated outputs using measures such as Shannon entropy and selectively reinforces those with lower entropy (i.e. higher self-confidence) (Prabhude- sai et al. 2025 ZZhao al. 2025 Zhang al. 2025b).
While these self-evaluation approaches aim to construct precise pseudo-labels by leveraging the rich knowledge of large pretrained models, recent studies (Chowdhury al., 2024) have highlighted the robustness of preference learning in enabling LLMs to improve even from noisy labels. This robust- ness calls into question the necessity of pursuing fine-grained eseudo-feedbace especially given that self-evaluation methods incur substantial computational overhead due to additional inference
steps and introduce overconfidence issues (Li et al. 2024a Zhang et al. 2025c) that stem from the model's 's intrinsic biases.
In this paper, we present a lightweight framework for self-improving models on unverifiable open- ended tasks, bypassing the need for self-evaluation mechanisms. Inspired by the principle of ma- jority voting, we introduce semantic voting, novel approach that relaxes the hard matching (i.e., exact matching) to soft matching based on semantic similarities. Specifically, each self-generated response is encoded into a semantic vector using a sentence embedding model, and its voting score is computed as the average cosine similarity with all other responses. Based on these scores, we iden- tify the most and least favored responses to form preference pairs, which serve as pseudo-signals to refine the model through preference learning.
Our experiments on machine translation and text summarization demonstrate that the semantic vot- ing framework consistently enhances performance across diverse model architectures and tasks, outperforming self-evaluation approaches in both stability and efficiency. The computational over- head of semantic voting amounts only a small fraction (ranging from several thousandths a few pereent) of the time required by self-evaluation baselines. Further analysis confirms that semantic voting provides a meaningful and effective unsupervised signal, going beyond what might otherwise be a "spurious reward" (Shao et al. 2025).
2 RELATED WORKS
2.1 LLM SHLF-IMPROVEMENT BY MAJORITY VOTING
Majority voting was initially introduced as decoding strategy to improve the test-time accu- racy (Wang et al. 2023). It decides the final answer by selecting the most frequent one among multiple generated candidates. Inspired by its success, LMSI (Huang al., (023) adapted majority voting as a criterion for rejection sampling to enable unsupervised improvement for LLMs. Subse- quent studies integrated it into alignment frameworks. ScPO (Prasad et al. 2024) builds preference
pairs with the most- and least-voted answers, using Direct Preference Opumization (Rafailov et al., 2023) for model improvement. TTRL (Zuo al. 2025) shapes rewards via majority voting and optimizes the model with GRPO (Shao et al. majority voting as part of their data filtrrng strategies or construction basis for pseudo-labels (Jiang et al. 20256] Diao et al., 2025 Zhang et al.) (025a). Despite the widely demonstrated effectiveness of majority voting, its application is strictly limited to tasks where candidate answers can be exactly matched, such as arithmetic or multiple-choice questions.
2024). There are also other approaches that leverage
2.2 LLM SHLF-IMPROVEMENT BY SEL.FJUDDGIN
Self Self-judging is rooted in the "LLM-as-a-Juugee" paradigm, which leverages LLMs as evaluators to provide cost-effective feedback for complex tasks (Zheng et al. 2023 Gu et al. 2024). In the context of self-improvemmen,, self-judging typically involves instructing the model to assess its own gener- ated responses, often by assigning numerical scores or binary decisions. These self-assessmenns are then used as pscudo-labels for data filtering as reward signals in training (Li et al. 2024b Yuan et al. 20224] Pang et al., 2024: Lee et al., 2024; Zhang et al. 2024b). While
offers a simple and response format- agnostic approach to handle diverse tasks, its impartiality can be under- mined by the self- preferrffee bias, which also increases vulnerability to reward hacking (Li et al.]
2024a). Moreover, self-judging presumes strong evaluative capabilities, limiting efficacy smaller
2

---

Published as a conference paper at ICLR 2026
or less capable models (Wang et al. 2025) The generation of detailed evaluation rationales also incurs non-negligible computational overhead, hindering large-scale use eLi al. 2024a).
2.3 LLM SELF-IMPROVEMENT BY ENTROPY MINIMIZATION
Recently, entropy minimization has emerged a popular self-improvement strategy for LLMs, encouraging models to favor high-confidence (low-entropy) outputs. [Agarwal et al. (2025) intro- duced token- and trajectory-level entropy as two effective RL rewards, demonstrating their efficacy in boosting reasoning performance. Concurrently, RENT (Prabhudesai et al. 2025) confirmed the token-level entropy's 's utility and found that restricting proves performance. INTUITOR (Zhao et al., 2025) instead uses self-certainty, the average KL divergence between the uniform distribution and the model's logits, reward. GenRM (Li et al. 2025) provides theoretical grounding for entropy minimization: from the Inverse Reinforcement Learning perspective, pretrained logits act as optimal Q-functions, making entropy minimization natural alignment with pretraining knowledge.
specific segments of outputs further im-
Despite its strong performance on reasoning tasks (e.g., math, coding), entropy minimization ex- hibits notable inconsistencies across different models and tasks. (Agarwal et al. (2025) observed that entropy minimization fails when task bias diverges from pretraining data, misleading confidence to- ward unhelpful priors. They also reported inconsistency across different models, suggesting heavy dependence on the base model's inherent capabilities, /Zhang et al. (2025c) further highlights the risk of overconfidence, showing that deuced entropy minimization may lead to performance degradation and even model collapse.
METHODOLOGY
In this section, we introduce our semantic voling-based self-improvement (SVSI) framework, which assigns pseudo-scores self-generated responses based semantic consensus and constructs pref- erence pairs for learning. An overview is shown in Figure [
Self-generation
Filtering by clustering
Semantic voting
27; "Pacopalm und Doden versinisen"
fan (of, a
Tranilae::
Facepalm and sink inno the ground
Froepulm und im Boden sinken
0.30
0.35
Flep and sink into the ground
LLM
a?
mman
a
Facepalm
0.74
Triin
04
0.97
[mis a )
Focepalm ground
Figure 1: Overview of semantic voting-based self- self-improvement (SVSI) for LLMs. Given an input question X,, SVSI first generates candidate answer set A, clustering to identify the most coherent (largest) subset C.max (Section 3.2). and then performs semantic voting within C.max using average semantic similarities (Section used for DPO training (Section Answers with the highest (a.") and lowest (a)) voting scores are
3.1 SEMANTIC VOTING
Given an input question x; and N stochastic responses sampled from an LLM M, the candidate answer set A, defined
=a F = = -M(-x)))
(1)
where y represents the generated response and of = a(y!) denotes the predicted answer parsed from y/- For simplicity, use af throughout.
3

---

Published as a conference paper at ICLR 2026
Similar to majority voting (Wang al. 2023 semantic voting leverages a consensus mechanism among candidate answers. but instead of hard exact matching, employs soft measure via a similarity function faim R [0, 1]:
Σ
fai (a)
Ai
df.af J..kk..
The answer with the highest Ssv score is thus the one most semantically aligned with the rest of the set, ie., the "most voted" under semantic consensus.
In this work, employ lightweight implementation of the similarity function fsim defined the cosine similarity between the sentence embeddings of two candidate answers:
Memb ( (a.) Memb ( (af)
Ssim (at, a²) |1Memb (at)l I| x |[Memb (a:)l
where Memb is a sentence embedding model that embeds the answers into semantic vectors.
3.2 SELF-GENERATION FILTERING
Intuitively, semantic voting is to identify semantic consensus among candidate answers and score each candidate by its alignment with such shared consensus. Therefore, to ensure meaningful and re- liable consensus formation, the candidate pool should exhibit relatively limited variation; otherwise, severely deviant samples may skew the voting outcome. While the candidate answers are generated in response the same question and thus expected to be basically analogous, remains possible for stochastic self-generation to produce outliers that diverge significantly from the majority. Our empirical observation also confirms that such anomalous candidates can undermine the effectiveness of semantic voting, particularly for less capable models.
To address this problem and facilitate the stability of self-improvement, we filter the self gelf-generated answers by clustering process before semantic voting. Specifically, apply a density-based clustering algorithm (Malzer Baum 2020) to aggregate the candidate answer set A, into one more clusters {C, Jx. with their sentence embeddings as features and the cosine similarity as distance. Subsequently, we retain only the largest cluster Cimas as the refined candidate set and discard all others. As result, semantic voting will only be applied answers af C.max:
Ssv c/maa
Σ
fsilla; a
(4)
1Cmmax
of aat ECimax ,Kjj
Algorithm 1: Building DPO-style training pairs
Input: Base LLM M; Unsupervised dataset D,; Sentence embedding model Memb; Sample size N; Clustering function fc
Output: Training dataset for DPO Dápo
for x, in D,, do
Generate N candidate answers; A, = la/la? M(-xa))
Aggregate candidate answers into answer clusters; (C/ 1 - (c((Mems(a))ld (a)mm A,)
Get the maximum answer cluster; C.max arg max 1,*1|;
Get the preferred answer; am + arg max Ssv (a/ (Cmas): of c/max
Get the dispreferred answer, af t- arg min Ssv (G,ax) of cmma
Add [xi, (x..a]" at" .a/) to Dapo:
eend
4

---

Published as a conference paper at ICLR 2026
wmr24ppp.e
wmrZ4pppJr
wmz2-ppp.n
wmr34pp.es
mmn.dulymaill
puhmed
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
BLEU
a-MQM
Rougel.
BLEURT
Rougel.
BLEURT
base
16.12
19.12
13.63
13.72
10.63
16.92
20.11
16,78
13.50
29.62
D
Qwen-1.5B
3.45
6:73
9.35
7.47
27.46
NA
18.21
20:20
19.77
19:19
11.57
18.45
24.32
20.16
1.63
4.41
NA
SVSI
18.04
20.53
20.34
19.96
11,93
18.48
23:46
19.88
13.72
30.63
NA
NA
2.91
9.39
4.29
9.10
1.71
8.27
5,47
8.85
16.86
37.93
27.62
50.23
Lhuma-18 1B
EM
1.51
8.68
4.35
8.50
4,86
11.11
5.21
8.63
14.45
25, 66.66
28. 14
45.69
SVSI
5.28
10.60
5.59
9.74
2.75
9.10
6.89
9.35
17.83
40.28
27.70
50.92
17.64
Qwen-3B
18.62
20.02
18.18
18.73
NA
NA
EM
19.32
20.53
19.72
17.67
13,66
17.57
25.22
19.65
NA
NA
NA
NA
SVSI
20.44
21.46
20.79
19.49
13.41
18.76
25.27
20.04
NA
NA
NA
27.77
Llama-38
EM
10.52
4.78
13.14
9.88
14,00
4.67
14,04
9.09
11.34
1,26
16.31
7.95
12.01
5.09
11.82
8.46
20.69
0.18
38,97
1,92
23.01
29.35
46.82
43.96
SVSI
8.71
11.59
10.51
11.60
5,42
10.53
11.89
10.80
9.18
20.36
27.67
50.09
16.42
19.71
16.67
11,62
15.31
23.67
16.31
16:44
33.9
18.81
32.44
Owen-7B
18.34
18.45
21.20
18.52
12.08
16.34
25.58
17.70
7.49
16.33
23.91
38.90
EM
7.32
10:05
17.66
14.18
11.63
14.95
17.01
11.48
17.65
33.75
30.98
46.
base
20.22
20.12
24.32
20.20
13.94
18.18
28.92
20.45
20.68
45.48
29.48
53.68
Llamn-88
EM
SJ
20.83
0
20.62
22.35
11.46
19.96
12.29
7.19
8.98
12.85
12.02
26.08
12.4
18.12
9.01
23.33
0.08
44.78
1.73
33.86
10.34
51.62
21.08
24.74
Table 1: Evaluation of SVSI against Self-Judging (SJ) and Entropy Minimization (EM) transla- tion (BLEU, n-MQM) and summarization (Rouge-L, BLEURT) tasks; "NA" indicates cases where base models failed generate valid outputs, rendering evaluation inapplicable.
3.3 TRAINING
Equation 4 assigns continuous that reflects the relative response quality. In principle, these pseudo-signals can be leveraged by various training paradigms, for instance, as reward signals in online reinforcement learning to build contrastive pairs in preference learning. However, such unsupervised signals inevitably con- tain noise, requiring robust training framework to ensure consistent performance gains. To this end, we adopt the Direct Preference Optimization (DPO) (Rafailov et al. (202)) in SVSI, since re- cent studies (Chowdhury et al., 2024) have demonstrated that DPO exhibits provable robustness to mislabeled or imperfect data, rendering it especially well-suited to our setting. To maximize the preference margin, we construct training pairs by selecting the candidate with the highest voting score as the preferred response an and the one with the lowest score as the dispreferred response a The full procedure for generating DtO-style training pairs is formalized in Algorithm [ Using the resulting dataset Dapo. optimize the model with the standard DPO loss (Rafailov et al. 2023):
each candidate answer, enabling fine- mme-ggained ranking
Ix(a" log Tee Are
Ta(a'(x) Tyes(al
LDPO = -B,r,"" ,at) [log o rB log
where or denotes the sigmoid function, B the regularization coefficient, and Rg and Aref rep- resent the parameterized training policy and reference policy, respectively. We further discuss the applicability of semantic voting to alternative training paradigms in Section 4.5
4
EXPERIMENTS
4.1
SETUP
4.1.1
DATASETS
We evaluate SVSI on two classical open-ended unverifiable NLP tasks: translation and summariza- tion. For translation, employed four subsets of WMT24++ (Deutsch et al.) 2025), involving translating from German, French, Russian, and Spanish into English. These subsets are denoted as wmr24pp_de _de, wmt24pp ffr, wmt24pp_ruu and wmt24pp respectively. For summarization. the CNN//ailymail dataset (cnn dailymail) (Hermann et al., al, (015), regarding news summarization, and the medical summarization dataset for PubMed abstracts (pubmed) (picetie) 2025).
5

---

Published as a conference paper at ICLR 2026
4.1.2 EVALUATION PROTOCOL
To enhance the comprehensiveness of the evaluation, we employ two kinds of metrics--lxical metric and semantic Saarii--mesuurnn different aspects of the answers. For translation, we use BLEU (Papineni et al. 2002) as the lexical metric and MetricX-24 (Juraska al.) 2024) to assess semantic quality. MetricX-24 is well-trained. reference-free evaluation model that provides auto- matic quality estimates the form of raw MQM ratings, where lower scores indicate better trans- lation quality. For consistency in interpretation, in the following sections, we report normalized MQM score n-MQM = TaRqu aligning the direction of improvement with that of other metrics.
For summarization, we adopt ROUGE-L Lin 2004) as the lexical metric and BLEURT (Sellam al., 2020) as the semantic metric. We do not use reference-free evaluation methods for summa- rization, as recent studies (Zhang et al. (2024a) show that such methods often struggle to accurately assess outputs generated by large language models.
4.2 IMPLEMENTATION DETAILS
We use six base models of varying scales: three from the Llama series (Dubey et al., (2024 (IB, 3B, and 8B parameters) and three from the Qwen series YYang et al., 2024) (1.5B, 3B, and 7B pa- rameters). During self- elf-generation, we sample 64 responses per input using temperature 0.7 and top-p of 0.9. For semantic voting, we use Sim- CSE (Gao et al. (021) to obtain sentence embed- dings and cluster generated responses via HDB- SCAN (Malzer & Baum 2020). For evaluation, answers are generated via greedy decoding. Both self-generation and evaluation use identical zero- shot prompts, and all training hyperparameters are held constant across methods. Full implementa- tion details are provided in Appendix [ and sen- sitivity study about clustering algorithms can be found in Appendix
SVSI
EM
Somantie di cenoovenie (%) 14444
imppon
Semantic
1
Lexical improvement (%)
Figure 2: Overall lexical and semantic im- provements of SVSI, EM, and SJ across all set- tings, relative to the base model performance.
4.3
MAIN RESULTS
In this section, we evaluate the effectiveness of SVSI through experiments conducted across all datasets and models. Three exceptions are ex- cluded: Qwen-1.5B on pubmed, Qwen-3B on daidammmaa,,, and Qwen-3B on pubmed, where the base models fail to provide any valid answers. We compare against two self-evaluation baseline approaches: self-judging (SJ) and entropy mini- mization (EM). For SJ, we follow the setup from SRLM (Yuan et al. 2024), instructing the model score its own generated responses on a 5-point scale criterion, and constructing preference pairs from the highest- and powest-graded outputs. For EM, we adopt the trajectory-level entropy (Agar- wal et al. 2025) as the measurement, selecting re- sponses with the highest and lowest entropy for preference learning.
Seconds
10
7B
Qwen Models
Seconds
10'
Llama Models
Figure 3: Computational overhead for building preference pairs via SVSI, EM, and SJ.
Table presents the results. As shown, SVSI consistently outperforms the base model across all settings. Compared to the two self-evaluation baseline methods, SVSI performs on par with or better the Llama-3B
than them in most cases, though it is less effective in certain instances, particularly model. To provide an overall comparison, we compute the improvement percentage of each method relative to the base model across all configurations and visualize the results in Figure 2 The x--axis

---

Published as a conference paper at ICLR 2026
wrzz-ppp.
wm224pp_ If Qwen-1.0
wmn24pp_m nu Qwee-1150
wiIZZApp__
cmn dailymail
pubmed
1.50
Liara-ill
Lima-ii
Lama--ill
Liara-BB
Limmslll
Locial
Qwen-10
Quen-B8
Qwee-33
Quic-Til
Quen-18 Qwensil
Quen-TRR Own-11
Am--B Quen-Sil
Limra-38
Llama-BB
1lare-3B
Llama-1B
Llama-38
Llams33
bac
fliped-SV
Qwen-1150
Qwen
Lian-BB
Liana-ii
Semurte
Qreen
Am-1B
Qwane-3l
Quin-Til
mmm-1 Quee-al
Qwm--i
Am--i Quen-ill
Limma33
Llama-B
Llama-3B
LlamaJBB
LIama-3
Llama3B
Figure 4: Improvements of training with Semantic Voting (SV) and flipped-SV (inverted SV pref- erence pairs), relative to the base model on lexical (top row) and semantic (bottom row) metries.
and y-axis represent improvements lexical and semantic metrics, respectively, with logarithmic scaling applied both axes. As the figure illustrates, while all three methods demonstrate significant gains some cases, SVSI exhibits more consistent and stable performance, with no instances of severe degradation. This advantage likely stems from its independence from self-evaluation, thereby avoiding the pitfalls of model overconfidence.
Another notable advantage of SVSI its significantly lower time overhead. We compute the average time required to construct preference pairs across all datasets for the three methods. The results, shown in Figure 3 reveal that SVSI requires orders of magnitude less time compared to the two baseline methods, and its computational cost does not scale with the size of the base model, making particularly efficient for larger models.
4.4 VALIDATION FOR SEMANTIC VOTING
Recent studies (Shao al. 2025) have shown that reinforcement learning with even a corrupted or spurious reward signal can significantly enhance reasoning capabilities in certain LLMs, perhaps be- cause triggers existing behaviors instilled during pretraining. This raises concerning possibility that the surface effectiveness of newly designed reward signals may be just an illusion. Therefore, to rigorously assess the true contribution of semantic voting, we conduct experiments on a fipped-SV setting, where we invert the preference pairs constructed by semantic voting and proceed with DPO training using these reversed signals. If semantic voting really captures meaningful preferences, can be anticipated that the flipped-SV would result in performance degradation.
Results are presented in Figure where the first and second row indicates improvements
lexical and semantic metrics, respectively, and each column cor- responds to different dataset. As shown, flppee-- consistently underperforms standard semantic vot- ing, and in nearly all cases also performs worse than the base model. In general, the extent of perfor- mance degradation flippe-SSV is proportional the degree of improvement achieved by semantic voting, which aligns with our expectations. While there are few isolated cases where flipped-SV still surpasses the base model, suggesting the presence of unreasonable improvement under spurious super- vision as described in (Shao et al. 2025), this effect too marginal to account for the primary gains ob- served; the main driver of improvements remains the valid signal captured by semantic voting.
EMIV
BLBU
wunt 1pp de
wmnz pp di
wimt1pp tpo. mu
wix2 Spp.
Qaven-1
1.58
BLEU
wint4pp de
wmi2-ipp
win14ppp m
wmx2ppp es
Llama-18
Figure 5: Evaluation of the GRPO-based variant SV (SVSI-G) against EMPO and EMRL-seq.

---

Published as a conference paper at ICLR 2026
4.5 EXTENSIBILITY ACROSS TRAINING METHODS
While we adopt DPO as our training framework in SVSI to improve robustness against noisy labels, semantic voting, in principle, assigns pseudo-labels to all self-generated responses, making it com- patible with alternative training paradigms. In this section, we evaluate its extensibility to Group Relative Policy Optimization (GRPO) (Shao et al., 2024) by integrating it directly as the reward signal (denoted SVSI-G). We compare against two GRPO-based baselines: EMRL-seq. which assigns rewards based on negative trajectory-level entropy (Agarwal et al. 2025), and EMPO, which first clusters generated samples and then rewards each response according the size of the cluster it belongs to (Zhang et al. 2025b). Results are presented in Figure 5
As shown, semantic voting with GRPO still demonstrates competitive average performance com- pared to the two baseline methods across four translation datasets. Nevertheless, both SVSI-G and EMRL-seq exhibit noticeable performance fluctuations across multiple runs, whereas EMPO shows markedly greater stability. We attribute this discrepancy to the form of reward signals. SVSI-G and EMRL-seq assign continuous scores to each sample, thereby constructing preference ranking with high cardinality. However, due to the inherent noise in these pseudo-rewards, many preference pairs within the ranking are erroneous, which impedes stable convergence. contrast, EMPO assigns discrete rewards and treats samples within the same cluster equally preferred, trading off reso- lution for stability. This suggests a direction for future work: incorporating semantic voting into hybrid strategies that preserve the semantic richness while enhancing training stability.
4.6 ABLATION STUDY
In our proposed framework, we apply a clustering procedure to the self-generated responses be- fore semantic voting. While this step is primarily intended to filter out interfering samples, thereby concentrating the candidate answers involved in voting, EMPO (Zhang et al. 2025b) has demonstrated that the clustering results also carry meaningful signals: samples in larger clus- ters tend to be of higher quality than those in smaller ones. To better understand the contribu- design
tion of clustering in our approach, SV", bypasses semantic voting and constructs preference pairs directly from the clustering re- sults, selecting the preferred sample from the largest cluster and the dispreferred sample from the smallest. The second, "w/o. clustering", removes clustering and performs semantic voting directly over all sene-gene samples. Results for both ablations are presented Figure As shown, removing semantic voting consistently leads significant performance drop, indicating that clustering results alone can not provide suf- ficiently reliable signals for effective DPO train-
ablation variants. The first, denoted as "w/o.
BLEU
ing. In contrast, the impact of removing clustering varies across models. For Diven-1.BB, performance remains stable, or clustering; while for Llama-1B, removing cluster- ing consistently degrades performance. This dis-
slightly improves, without
Qwen-1
crepancy likely stems from the difference in base model capabilities: Llama-IB produces a higher proportion of noisy off-topic responses, making clustering essential ensure semantic voting oper- ates on more coherent and meaningful samples. In contrast, Qwen-1.5H generates predominantly high- quality, low-noise responses during self- self-generation, rendering clustering redundant.
BLEU
went4pp_de de
wmp24pp.4 ft
WmD4pp_rn
wm24gppee
Clama: IB
Figure Ablation results for clustering and semantic voting (SV) in SVSI.
4.7 HYPERPARAMETER STUDY
In this section, examine the influence of key hyperparameters on final performance across two critical stages of SVSI: the self-generation stage, where the temperature and sampling size jointly determine the candidate answer set (Figure ?] and the filtering stage. where parameters of the clustering algorithm govern the filtering results, thereby shaping the construction of preference pairs (Figure 8). All experiments conducted on the wm/24pp de dataset.
8

---

Published as a conference paper at ICLR 2026
Imprectente Pl
Lama-A
10
-19
12
-10
Sampling
16
Imprremnns Mlg
Memo s-MMM Medel Lammo-111
Motric
MUM Model Qwan-1 1.50
Matric MQM Model - Llmma-315
Metric
MQM Model Owen 31----
32
64
10
Temperature
Temperature
Figure 7: Relative improvements compared to base models across different generation parameters: temperature and the sampling size.
Metric BLEE Medel Limma-BB
Metric BLEU Modd Owar- BLEU Moddl Liam--3B
Metric - BLEU Model Owen-B
Improveniets Imperemente
81
Metric n-M Modd - Uhma-18
Metricn--Mee -MM Modd Owens
Metric - =MMQ Madd -1 1mm-18
Matic mMM Mode Quan-8
2.5
Figure 8: Relative improvements compared to base models across two different clustering parame- ters of HDBSCAN: minimum cluster size (m) and saabbbrrhooddefiiing sample threshold (k).
Figure 7 presents the effects temperature and sampling size. We evaluate four sampling sizes (ranging from 16 to 128) and six temperature values (from 0.3 to 2.0). As shown, larger sampling sizes generally yield more stable performance across varying temperatures. In contrast, smaller sam- pling sizes (e.g., 16) introduce greater stochasticity into the candidate pool, resulting in increased performance fluctuation. As for temperature, values below 1.0 yield better results than higher ones. Excessively high temperatures tend to degrade response coherence, thereby disrupting cluster for- mation and potentially causing algorithmic failure. Model capacity also plays an important role: Qwen models demonstrate greater resilience to high temperatures than Llama models, and larger models overall exhibit improved stability compared to their smaller counterparts. Based on these observations, we recommend setting the temperature between 0.7 and 1.0, and using a sampling size of at least 32 to ensure reliable performance.
Figure[Xpresente the impact of clustering hyperparameters on model performance. The clustering al- gorithm adopted in SVSI, HDBSCAN (Malzer & Baum] 2020), is governed by two key parameters: the minimum cluster size m and the minimum number of samples required to define a neighborhood k. We evaluate four values of (ranging from 3 to 6), and for each, we test all valid k values satisfying & < m. The results indicate that larger values of m generally yield better overall perfor- mance. This can be attributed to that smaller m values tend to fragment candidate responses into an excessive number of tiny clusters, thereby compromising clustering coherence and diminishing the number of semantically meaningful, well-structured groups. Moreover, when becomes less sensitive to variations in k, suggesting enhanced robustness within this regime. In light of these findings, we recommend selecting larger values in practice achieve more stable and effective clustering results.
5, performance
4.8 SENSITIVITY STUDY ON EMBEDDING MODELS
In this section, we investigate the sensitivity of semantic voting to the choice of sentence embedding model. Specifically, experiment with BGE-en (Xiao al. 2023), DeBERTa-v3 (He al. (023)) and the multilingual BGE-mt (Chen et al. 2024), in addition to SimCSE, and summarize their downstream performance in Table

---

Published as a conference paper at ICLR 2026
wmt224pp.de
wmt24pp_frr
wmm24pp_r
wmt24pp.co
avg.
BLEU
n-MQM
BLEU
a-MQM
BLEU
u-MQM
BLEU
n-MQM
BLEU
n-MQM
base
16.12
19.12
13.63
13.72
10.63
16.92
20.11
16.78
15.12
16.63
D
18.03
20.58
20.26
19.57
12.29
18.73
23.40
19.92
18.50
19.70
Qwen-1.58
BGE-en
19.97
19.65
12.13
22.36
19.19
BGB m3
16.65
20.41
19.83
19.27
11.51
18.12
23.21
19.23
17.80
19.26
SimCSE
18.04
20.34
19.96
11.93
23.46
19.88
encoder
18.35
20.62
19.82
19.53
11.92
18.80
23.14
19.69
18.31
19.66
base
2.91
9.39
4.29
9.10
1.71
8.27
5.47
8.85
3.60
8.90
V3BERTaV3
3.11
4.48
9.26
1.54
7.96
5.01
8.55
3.54
Llama-IB
BGE
4.60
9.98
4.96
9.35
2.47
9.29
7.01
9.54
4.76
9.54
BGB-m3
5.15
10.37
4.19
8.96
1.47
8.24
7.66
9.92
4.62
9.37
SimCSE
5.28
10.60
9.74
2.75
9.35
5.13
9.70
cross encoder
4.49
10.15
4.45
9:42
1.18
7.82
5.18
8.56
3.83
8.99
Table 2:
Performance comparison among different semantic similarity computing approaches.
As the results indicate, all embedding models achieve nearly identical performance when applied to the Qwen-1.5B model. However, when applied to the Llama- model, DeBERTa-v3 exhibits sig- nificant performance drop on certain tasks, leading to a notably lower overall score. We hypothesize that this discrepancy stems from the limited capacity of Llama-1B, which yields less concentrated output distribution. This, in turn, amplifies variance at the embedding level across different mod- els, rendering voting results more sensitive. Another noteworthy observation is that, despite being trained multilingually, BGE-m3 does not outperform monolingual embedding models, suggesting that multilingual capability is orthogonal to the effectiveness of semantic voting.
Beyond cosine similarity based on sentence embeddings, cross-encoders (Reimers & Gurevych, 2019) offer an alternative approach measuring semantic similarity between text pairs. To assess their suitability within SVSI, we employ macro-Tiny BERT The corresponding results are also re- ported Table 2. Overall, cross-encoders perform comparably to embedding based methods. Their main drawback lies in computational cost, necessitating N2 forward passes since cross-encoders require pairwise inputs. Despite this limitation, we believe cross-encoders represent promising avenue for future exploration, particularly when adapting SVSI to long-text scenarios.
5 CONCLUSION
In this paper, we propose semantic voting, highly efficient, self-evaluation-free metric designed for self-improvement of large language models on unverifiable open-ended tasks. Instead of rely- ing computationally expensive and potentially biased self-evaluation methods, semantic voting leverages a lightweight sentence embedding model to encode self-generated responses and assigns pseudo-labed to each response based on its average semantic similarity with the others. Comprehen- sive experiments across various models and datasets demonstrate that semantic voting consistently matches surpasses the performance of self-evaluation baselines, while reducing computational overhead by orders of magnitude (as low as merely 0. 0.1% 1%-5% of the original expense). Further anal- ysis confirms that semantic voting offers simple, effective, and scalable approach to facilitate large language models self- seff-improvement.
ACKNOWLEDGMENTS
This research was supported by Theme-based Research Scheme (T45-205/21-N) from Hong Kong RGC, and Generative AI Research and Development Centre from InnoHK.
REPRODUCIBILITY STATEMENT
More implementation details are provided in Appendix B All datasets used are publicly available, and the official code is released at http::///gtttooooooooooooooooott
ctt
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. The unreasonable effec- tiveness of entropy minimization in LLM reasoning. CoRR, abs/2505. 15134, 2025.
Sina Alemohammad, Josue Casco- -osso-Rodr,,,e, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaci, Daniel LeJeune. Ali Siahkoohi, and Richard Baraniuk. Self-consuming generative mod- els go mad. In The Twelfth International Conference on Learning Representations, 2023.
Mihael Ankerst, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. OPTICS: ordering points to identify the clustering structure. SIGMOD Conference, pp. 49-60 ACM Press, 1999.
Loic Barrault, Ondrej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Gra- ham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. Findings of the 2019 conference on ma- chine translation (WMT19). In WMT (2), pp. 1-61. Association for Computational Linguistics, 2019.
Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Kochn, Johannes Lev- cling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Spe- cia, and Ales Tamchyna. Findings of the 2014 workshop statistical machine translation. In WMT@ACL, pp. 12-58. The Association for Computer Linguistics, 2014.
Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-embedding: Multi-linguality, multi-functionallty, multi guanuliganul text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 2318- 2335. Association for Computational Linguistics, 2024.
Yuri Chervonyi, Tricu H. Trinh, Miroslav Olsák, Xiaomeng Yang, Hoang Nguyen, Marcelo Mene- gali, Junehyuk Jung, Vikas Verma, Quoc V. Le, and Thang Luong. Gold-medalist performance in solving olympiad geometry with alphagcometry2. CoRR, abs/2502.03544, 2025.
Sayak Ray Chowdhury, Anush Kini, and Nagarajan Natarajan. Provably robust DPO: aligning language models with noisy feedback. In ICML. OpenReview.net, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat. Noveen Sachdeva, Inderjit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan- Jiang Jiang, Krishna Haridasan, Ahmed Omran, Nikunj Saunshi, Dara Bahri, Gaurav Mishra, Eric Chu, Toby Boyd, Brad Hekman, Aaron Parisi, Chaoyi Zhang, Kornraphop Kawintiranon, Tania Bedrax-Weiss, Oliver Wang, Ya Xu, Ollie Purkiss, Uri Mendlovic, Πai Deutel, Nam Nguyen, Adam Langley, Flip Korn, Lucia Rossazza, Alexandre Ramé, Sagar Waghmare, Helen Miller, Nathan Byrd, Ashrith Sheshan, Raia Hadsell Sangnie Bhardwaj. Pawel Janus, Tero Rissa, Dan Horgan, Sharon Silver, Ayzaan Wahid, Sergey Brin, Yves Raimond, Klemen Kloboves, Cindy Wang, Nitesh Bharadwaj Gundavarapu, Ili Shumailov, Bo Wang, Mantas Pajarskas, Joe Hey- ward, Martin Nikoltchev, Maciej Kula, Hao Zhou, Zachary Garrett, Sushant Kafle, Sercan Arik, Ankita Goel, Mingyao Yang, Jiho Park, Koji Kojima, Parsa Mahmoudich, Koray Kavukcuoglu, Grace Chen, Doug Fritz, Anton Bulyenov, Sudeshna Roy, Dimitris Paparas, Hadar Shemtov, Bo- Juen Chen, Robin Strudel, David Reitter, Aurko Roy. Andrey Vlasov, Changwan Ryu, Chas Le- ichner, Haichuan Yang, Zelda Mariet, Denis Vnukov. Tim Sohn, Amy Stuart, Wei Liang, Minmin Chen, Praynaa Rawlani, Christy Koh, JD Co-Reyes, Guangda Lai, Praseem Banzal, Dimitrios Vytiniotis, Jieru Mei, and Mu Cai. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. CoRR, abs/2507.06261 2025.
Dorin Comaniciu and Peter Meer. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell, 24(5):603-61 619, 2002.
AL, Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu. Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan,
11

---

Published as a conference paper at ICLR 2026
Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. I.. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, J. J. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. Deepseek-rl: Incentivizing reasoning capability Ilms via reinforcement learning. CoRR, abs/2501. 19488, 2025.
Daniel Deutsch, Eleftheria Briakou, Isaac Rayburn Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Eliz- abeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, and Markus Freitag. WMT24+++ expanding the language coverage of WMT24 to 55 languages & dialects. ACL. (Findings), pp- 12257-12284. Association for Computational Linguistics, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang. Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi. Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The Ilama 3 herd of models. CoRR, abs/2407.21783 2024.
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled al- pacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04477. 2024.
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, pp. 226-231. AAAI Press, 1996.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simese: embeddings. In EMNLP (1), pp. 6894-6910, Association for Computational Linguistics, 2021.
Simple contrastive learning of sentence
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Yuanzhuo Wang, and Jian Guo. A survey on llm-as-a-judge. CoRR, Abs/2411.11594 2024.
Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. The curious decline of linguistic diversity: Training language models synthetic text. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 3589-3604, 2024.
Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangle embedding sharing. In ICLR. OpenReview.net, 2023.
Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pp. 1693- 1701, 2015.
12

---

Published as a conference paper at ICLR 2026
Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In EMNLP, pp. 1051-1068. Association for Computational Linguistics, 2023.
Chunyang Jiang, Chi-Min Chan, Yiyang Cai, Yulong Liu, Wei Xue, and Yike Guo. Graceful forget- ting in generative language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 13165-13180, 2025a.
Chunyang Jiang, Chi-Min Chan, Wei Xue, Qifeng Liu, and Yike Guo. Importance weighting can help large language models self-immproe. In AAAI, pp. 24257-24265. AAAI Press, 2025b.
Fangkai Jiao, Geyang Guo, Xingxing Zhang, Nancy F. Chen, Shafiq Joty, and Furu Wei, Preference optimization for reasoning with pseudo feedback. In ICLR. OpenReview.net, 2025.
Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. Metricx-24: The google submission to the WMT 2024 metrics shared task. In Barry Haddow, Tom Koemi, Philipp Koehn, and Christof Monz (eds.), Proceedings of the Ninth Conference Machine Translation, WMT 2024, Miami, FL, USA, November 15-16, 2024, pp. 492-504, Association for Computational Linguisties, 2024. doi: 10. 11118//11/22224, http:/2024..TT--- 1.35. /ttp://tt://///..rr/// 18653/v11 2024. 11224.wt-1... wmt-1.35
Tom Koemi and Christian Federmann. Large language models are state-of-the-ar evaluators of translation quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Trunslation, pp. 193-203, 2023.
Sangkyu Lee, Sungdong Kim, Ashkan Yousefpour, Minjoon Sco, Kang Min Yoo, and Youngjae Yu. Aligning large language models by on-policy self-judgment. In ACL (1), pp. 11442-11459. Association for Computational Linguistics, 2024.
Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. From generation to judgment:
Opportunities and challenges of lmm-as-a-judge. CoRR, abs/2411.16594,
2024a.
Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. Self- Self-alignment with instruction backtranslation. In ICLR. OpenReview.net, 2024b.
Yi-Chen Li, Tian Xu, Yang Yu, Xuqin Zhang, Xiong-Hui Chen, Zhongxiang Ling, Ningjing Chao, Lei Yuan, and Zhi-Hua Zhou Generalist reward models: Found inside large language models. CoRR, abs/2506. 233335, 2025.
Chin-Yew Lin. ROUGE: package for automatic evaluation of summaries. In Text Summarization Branches Out. pp. 74-81, Barcelona, Spain. July 2004. Association for Computational Linguis- tics. URL /ttpittt//////aahoooo..... 0rt. 10
Aixin Liu, Bei Feng. Bing Xue, Bingxuan Wang. Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 1rX44:24/2.19437, 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR (Poster). Open- Review.net, 2019.
Claudia Malzer and Marcus Baum. A hybrid approach to hierarchical density- cluster selec- tion. In MFI, pp. 223-228 IEEE, 2020.
Jing-Cheng Pang. Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. Language model self improvmproee by reinforcement learning contemplation. In ICLR. OpenReview.net, 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pp. 311-3111-.,.. ACL, 2002. doi: 10.3115511733033.07333 URL http://ss//ccc.aaooooooooorrr//p
13

---

Published as a conference paper at ICLR 2026
pieetie. pubbecd-absstrat--uumnaay Hugging Face Dataset, 2025. URL https::/hugginngfaee
Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Maximizing confidence alone improves reasoning. CoRR, (ass/2505.22660, 2025.
Archiki Prasad, Weizhe Yuan, Richard Yuanzhe Pang, Jing Xu, Maryam Fazel-Zarandi, Mohit Bansal, Sainbayar Sukhbaatar, Jason Weston, and Jane A. Yu. Self-consistency preference op- timization. CoRR, 11122411044099 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. In NeurlPS, 2023.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In EMNLP-IJCNLP, pp. 3982-3992. Association for Computational Linguistics, 2019.
Thibault Sellam, Dipanjan Das, and Ankur P. Parikh. BLEURT: learning robust metrics for text gen- eration. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 7881- Association for Computational Linguistics, 2020. doi: 10.18653/ V1/2020,CLL-MANN 704. htttp:://///o..... /ttps///66booor///.1166///////2 .al9:2000c1l1100 acl-main. 704
18653/v1/2020
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng. Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi. Pang Wei Koh, and Luke Zettlemoee. Spurious rewards: Rethinking training signals in RLVR. CoRR, abs/2506. 10947, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.033000, 2024.
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631(80222 2024.
Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn. Position: Will we run out of data? limits of LLM scaling based human-generated data. In ICML. OpenReviennne.e 2024.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning language models. In ICLR. OpenReview.net, 2023.
Zhaoyang Wang, Weilei He, Zhiyuan Liang, Xuchao Zhang, Chetan Bansal, Ying Wei. Weitong Zhang, and Huaxiu Yao. CREAM: consistency regularized self-rewarding language models. In ICLR. OpenReview.net, 2025.
Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah Smith, Mari Ostendorf, and Hannanch Hajishirzi. Fine-gnt Eine-grained human feedback gives better rewards for language model training. In NewrIPS, 2023.
Yu Xia, Rui Wang. Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian J. McAuley, and Shuai Li. Beyond Shain-offthhouhhe A survey of chain-of-x paradigms for 1lms. In COLING, pp. 10795- 10809. Association for Computational Linguistics, 2025.
Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding. CoRR, fbbs/2309.07597, 2023.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu. Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu
14

---

Published as a conference paper at ICLR 2026
Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. CoRR, abs/2412.15115, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge. Haoran Wei, Huan Lin. Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang. Jian Yang, Jiaxi Yang, Jingren Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin. Xingzhang Ren, Xinyu Wang, Xinyu Zhang. Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. CoRR, abs/2505.0933 2025.
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models. In ICML. OpenReview.net, 2024.
Kongcheng Zhang. Qi Yao, Shunyu Liu, Yingjie Wang, Baisheng Lai, Jieping Ye, Mingli Song, and Dacheng Tao. Consistent paths lead to truth: Self-rewarding reinforcement learning for L.M reasoning. CoRR, abs/2506.00745. 2025a.
Qingyang Zhang, Haitao Wu, Changqing Zhang, Peilin Zhao, and Yatao Bian. Right ques- tion is already half the answer: Fully unsupervised LLM reasoning incentivization. CoRR, abs/2504.05811 2025b.
Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen R. McKeown, and Tatsunori B. Hashimoto. Benchmarking large language models for news summarization. Trans. Assoc. Com- put. Linguistics, 12:39-57, 2024a.
Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Lifeng Jin, Linfeng Song, Haitao Mi, and Helen Meng. Self-alignment for factuality: Mitigating hallucinations in ilms via self-evaluation. In ACL (1), pp. 1946-1965. Association for Computational Linguistics, 2024b.
Yanzhi Zhang, Zhaoxi Zhang, Haoxiang Guan, Yilin Cheng, Yitong Duan, Chen Wang. Yue Wang, Shuxin Zheng, and Jiyan He. No free lunch: Rethinking internal feedback for LLM reasoning. CoRR, abs/2506. 0712506.172111 17219, 2025c.
Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards. CoRR, abs/2500.11590, 2025.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In NeurlPS, 2023.
Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biging Qi, Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. TTRL: test-time reinforcement learning. CoRR, abs/2504.1 2025.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A THE USE OF LARGE LANGUAGE MODELS
In this work, we utilize large language models (LLMs) solely as writing assistants to identify gram- matical errors and refine linguistic expressions for improved clarity and fluency.
B IMPLEMENTATION DETAILS
The concrete versions of Llama models we adopted in the experiments are: 2-1e-33.-1B-IItruce
and 1amma---8B-Instree and the versions of Qwen models are: Qwen-2. 2-5m- 58-Instruct, Qwen-2. 1122.555---nnnue Instrruct, and Owen-2.5-7B-1 2.nn-..t-B-Instruct..
During the self self-generation phase, for each input, we sample 64 responses using a temperature of 0.7 and top-p of 0.9. All generations prompted in zero-shot setting. To facilitate automated parsing, we instruct models to enclose their final answers within (boxed()) Responses that fail to adhere this formatting convention excluded from preference pair construction. The prompt templates used for each task are illustrated in Figures 10 and
Prompt Template for wmt24pp de/fr/ru/es
You translation assistant who carefully and thoughtfully translates German/F cench/Russian///S
sentences into English, ensuring that the translated sentences fluent and accurately convey the original meaning. Place your translation answer in (boxed For axmaple, the answer is "Hello World", you should output 'boxed ( (Hlllo World).
User:
The Geerman/Frende French/ Russian /Saiiinh sentence is: '<ouuree sentenees! Please think about how to translate step by step.
Figure 9: Prompt template for four wmt24pp translation tasks.
Prompt Template for can dailymail
You skilled summarization assistant. Given concise, informative summary in three sentences, totaling around 50 words. Please provide your summary in plain text and place inside 'boxed (). For example, if your summary is "Hello World", you should oulput: \boxed (Hello World}
report, provide
User:
Here is the news article:
<news text>
Please summarize the article in three sentences.
Figure 10: Prompt template for the can.yymlyaail task.
For semantic voting, employ SimCSE (Gao et al. 2021
obtain sentence embeddings and
apply HDBSCAN (Malzer & Baum 2020 for clustering. Unless otherwise specified, we fix two key hyperparameters of HDBSCAN: the minimum cluster size (m) is set to 5, and the minimum number of samples required to define a neighborhood (k) is set to 2.
16

---

Published as a conference paper at ICLR 2026
Prompt Template for pubmed
You skilled summarization assistant. Given medical article, provide concise, informative summary in one or two sentences, totaling around words. Please provide your summary in plain text and place it inside (boxed For example, if your summary "Hello World", you should output: bboxed (Hello World]
User:
Here is the medical article:
<medical text>
Please summarize the article in or sentences.
Figure 11: Prompt template for the pubmed task.
wmt224p..d
wm124pp.ft
wmt24pp.nu
wmt24pp.es
avg.
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
base
16.12
19.12
13.63
13.72
10.63
16.92
20.11
16.78
15.12
16.63
Qwen-1.5B
MS
18.54
20.28
20.59
19.76
12.23
18.59
24.79
20.04
19.04
19.67
OPTICS
18.52
20.08
20.87
19.80
12.48
18.55
24.45
20.12
19.08
19.64
HDBSCAN
18.48
base
2.91
9.39
4.29
9.10
1.71
8.27
5.47
8.85
3.60
8.90
Llama- IB
MS
3.86
9.71
5,51
9.32
1.20
7.76
4.83
8.40
3.85
6.93
OPTICS
2.06
8.22
HDBSCAN
5.28
10.60
5.59
9.74
2.75
9.10
6.89
9.35
5.13
9.70
Table 3: Comparing performance among different clustering algorithms.
During DPO training, we use the AdamW optimizer (Loshchilov & Hutter 2019) with a learning rate of 1 10-6. The DPO regularization coefficient set 0.4. Training is conducted with per-device batch size of for total of 4 epochs.
At evaluation, we generate responses using greedy decoding with maximum output length of 512 tokens. An exception is made for Qwen2. S--B-Innstrun whose reasoning traces tend to be sub- stantially longer. To reduce parsing failures due to truncation, we extend the maximum generation length to 800 tokens for this model specifically.
All evaluation datasets are publicly available 4 Due computational constraints, we use a subset of 1, 000 samples for cnn dailymail and pubmed datasets, rather than the full original sets. Codes for building these subsets are provided in the supplementary materials.
C SENSITIVITY STUDY ON CLUSTERING ALGORITHMS
In this section, we evaluate the robustness of SVSI with respect to different clustering algorithms. In addition to HDBSCAN (Malzer & Baum 2020), employ two alternative density-based methods:
OPTICS Ankerst et al. 1999) and Mean Shift (MS) (Comaniciu & Meer 2002). The corresponding results are shown in 3
As shown in the table, OPTICS achieves performance nearly identical to that of HDBSCAN across both the Qwen-1.51 and Llama- 1B models. In contrast, MS underperforms relative these two methods on Llama-1B. This similarity between HDBSCAN and OPTICS is likely attributable to their shared foundation in the DBSCAN (Ester et al., 1996), which leads to analogous clustering behaviors. Nevertheless, despite the relatively weaker performance than the other two methods, MS still yields improvements over the base model. This underscores again the importance of applying clustering based filtering step to self-generated responses prior to semantic voting for less capable models, which is also demonstrated in Section 4.6
>https//huggingffrccco hutns///hgennnteee..vaaseeseeeen co//datsee//aiiee/cnn..lliyyyy
httms///ungiiggaeeoooootsstsoggu coddataeer//oogll//w222pp
thars://huggnngffcc.co http:///ww.....cooooo///////oo/.
17

---

Published as a conference paper at ICLR 2026
wmi19, dd
wmt14_fr
wmt19. .ru
wmt14_cc
avg.
BLEU
n-MQM
BL.U
n-MQM
BLEUU
n-MQM
BL.EU
n-MQM
BL.U
n-MQM
base
25.95
19.69
19.45
14.73
16.93
18.18
12.06
19.01
18.60
17.90
Owen-1.5B
SVSI
27.06
19.88
25.27
20.58
18.55
19.76
12.78
20.45
20.91
20.17
base
9.02
9.29
11.03
9.99
4.36
7.97
4.83
9.15
7.31
9.10
Llama-11 IB
SVSI
11.96
10.70
9.64
9.74
6,06
9.10
5.10
9.43
8.19
9.74
Table 4: Out-of-distribution evaluation for SVSI on translation tasks.
D OUT-OF-DISTRIBUTION EVALUATION
In Section we evaluate the effectiveness of semantic voting under a test-time setting, where the model demonstrates performance gains on the test set through unsupervised tuning. A natural follow-up is whether such improvements generalize to unseen, out-of-distribution data. To address this, we assess the improved models on unfamiliar datasets, without any adaptation, fine-tuning, or even unsupervised recalibration.
Specifically, we employ wmt19 de, wmm19 ru, wmt14 fr. and wmil4 es (Barrault al. 2019 Bo- jar et al., 2014) to evaluate the models trained on wmt24pp de, wmt24pp wmr244p. and wmt24pp- es correspondingly. Results are summarized in Table 4 shown, although performance gains are not universal across all cases, semantic voting consistently improves results on the majority of unseen datasets. This demonstrates that the capability enhancement conferred by semantic voting is transferable beyond the original data.
E INSTRUCTION FOLLOWING EVALUATION
Llama-1B
A1
36.5
Llama-3B
13/8
Llama-8B
394
A.A
Qwen-1.5B
33/y
31.9
Qwen-3B
37,0
Qwen-7B
A9//
38.9
38,2
36.3
Win Rate (%)
Figure 12: Instruction following ability improvement with SVSI.
In this section, investigate the effectiveness of SVSI broader range of open-ended tasks. We evaluate its performance on AlpacaEval (Dubois et al. 2024). a benchmark for general instruc- tion following, using DeepSeek-V3. (Liu et al. (024)) as the judge LLM. Specifically, compare model performance before and after applying SVSI. The results, presented as win rates in Figure [2 demonstrate that SVSI can still yield general improvements across the instruction-followiing tasks.
F BACK-TRANSLATION EVALUATION
In Section we evaluate translation tasks by translating multiple languages into English. Given the inconsistency of translation capabilities across different languages (Koemi & Federmann 2023 Jiang et al., 2025a). in this section, we investigate whether SVSI remains effective when applied to back-translation. i.c, translating English into other languages. To support evaluation across
18

---

Published as a conference paper at ICLR 2026
wmf24pp.de_in
wmi24pp_ in
wm124pp.ru.a
wm124pp_es.hm
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
base
12.18
16.53
17.44
14.45
6.51
13.07
20.39
14.75
Qwen-7B
SVSI
15.22
20.58
19.19
15.72
8.75
15.11
22.98
16.67
base 16.09
23.04
23.34
19.01
9.90
14.84
26.29
18.38
Llama-8B
SVSI
17.51
23.70
23.97
19.27
10.57
15.80
27.16
18.94
Table Performance of translating English back other languages.
wmi24pp-de
wmt24pp_fit
wm124pp-ru
wmi24pp-es
calldmilymai
pubmed
base
86.36
86.17
86.01
86.04
79.61
86.86
TTR
SVSI
86.08
86.09
85.85
85.78
78.16
85.56
base
22.38
22.76
24.41
22.03
33.20
32.60
RSMD
SVSI
17.82
17.91
21.16
17.40
29.66
27.26
Table 6: Linguistic diversity changes after SVSI.
variety of languages, replace SimCSE with BGE-M3 (Chen et al. 2024),
multilingually trained
embedding model. As shown in Table 3 SVSI continues to deliver consistent improvements this back-translation setting.
G SVSI-G DISCRETIZED VOTING SCORE
we explore the extensibility of our approach across training methods by integrating mantic voting into the GRPO algorithm, resulting in the variant SVSI-G. While SVSI-G achieves petitive performance than baseline methods, it ex- hibits fluctuations. We hypothesize that this insta- bility stems from the nature of the reward signal. Specifically, the use of continuous scores leads to high-cardinality ranking space. To test this hy- pothesis, we evaluate SVSI-G with coarse- grasee rewards. Concretely, ing scores onto a 5-point scale by assigning the top 20% of scores to 5, the next 20% to 4, and so We refer to this discretized version as SVSI-G (dis- crete). As shown in Figure 13 SVSI-G (discrete) exhibits substantially reduced performance fluctua- tions compared the original SVSI-G in Figure 5 which supports our hypothesis.
In Section 4.5
EMPO
EMRL
BLEU
wwm24pp.com
win24pp ft
wim24pp
wmc24pp. 6
Qwen-1.c
BLEU
discretize the original vot-
writ-4pe.s de
wmi24pp
wmDD4ye.rv nu
wm24gpps
Tlama
IB
Figure 13: Evaluation of GRPO-based vari- ant of SVSI with discretized voting score (SVSI-G-diserete).
H STUDY ON LINGUISTIC DIVERSITY
For open-ended tasks, beyond quality, creativity is also crucial practical consideration. this sec- tion, we examine how linguistic diversity is affected by SVSI. To quantify this, employ the Type- Token Ratio (TTR) as a measure lexical diversity and the root mean square deviation (RMSD) of semantic embeddings
proxy for semantic diversity. Table X presents the results Qwen-7B. As shown, SVSI leads to reduction linguistic diversity. In general, semantic diversity declines more substantially than lexical diversity, and the drop is more pronounced in translation tasks than in summarization tasks.
This observation aligns with actively studied challenge in self teff-improvement frameworks, called model collapse (Guo et al., 2024; Shumailov et al. 2024 Alemohammad et al. 2002), where gener- ative models, when trained iteratively their self siff-generation, gradually narrow their output distri-
19

---

Published as a conference paper at ICLR 2026
wm
wmi24pp.ffr
wm
wmi24pp-es
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
BLEU
n-MQM
SVSI
18.04
20.53
20.34
19.96
11.93
18.48
23.46
19.88
Qwen-1.55
SVSI-hn
14.07
16.39
17.44
16.86
11.77
18.18
19.62
17.21
base
5.28
10.60
5.59
9.74
2.75
9.10
6.89
9.35
Llama-1B
SVSI-hn
7.02
11.79
6.31
10.62
4.24
10.70
10.04
11.57
Table 7: Evaluation of using hard negative samples in SVSI.
wmt24pp
wmi24pp
wmt24pp
wmt24pp es
SV-ranking 18.22
21.28
10.24
18.02
Qwen-7B
random-ranking
1.55
1.25
0.46
0.56
SV-ranking rundom-ranking
23.42
23.88
17.69
24.02
Llama-8B
0.32
0.87
0.63
0.78
Table 8: Kendall's tau correlation of semantic voting-based ranking against reward-based ranking.
bution due to the absence of external information, eventually leading to incapability. However, we believe that this is unlikely to pose a serious practical concern, as multi-turn self-training of gener- ative models is rarely employed in real-world settings. Moreover, if diversity in generated text is primary objective, the simplest, most direct, and highly effective solution to increase the sampling temperature.
STUDY ON UTILIZING HARD NEGATIVE SAMPLES
As discussed in Section 4.6 meaningful comparison sig- nals across different clusters are difficult to obtain. How- ever, certain hard negative samples, those responses with incorrect format, can be identified. While it is reason- able to assume that treating such samples as dispreferred examples and contrasting them with well-formatted re- sponses could improve the model's 's adherence to format- ting constraints, we deliberately excluded them in the standard SVSI formulation. By the way, we aim to elim- inate confounding effects arising from formatting errors and to isolate the impact of semantic voting on genuine semantic quality.
Metric Rouge-L Model Llama-1
Improvements, (%) Improvements (%6)
-20
Metrie BLEURT Model Llama-
6
-20
-40
In this section, we compare this design choice against alternative that explicitly leverages hard negative sam- ples. We denote this variant as SVSI-hn and evaluate it against the standard SVSI on Qwen-1.5B and Llama- Results are presented in Table 7 As observed, incorpo- rating hard negative yields better performance on Llama- 1B but consistently underperforms on Qwen-1.5B. We hypothesize that this discrepancy arises because SVSI-hn concentrates the learn ing direction toward format compliance rather than semantic distinction. This bias can be beneficial for weaker models that frequently violate formatting instructions, but for capable models like Qwen-1.55 1.5B, which already follow formatting reliably, such focus may distract from deeper seman- tic refinement, ultimately degrading performance.
Figure 14: Hyperparameter study on cnn/dailymail.
ANALYSIS ON SEMANTIC VOTING-BASED RANKINGS
In this section, we assess the consistency between rankings derived from semantic voting scores and the true underlying quality. As proxy for ground-truth quality, employ the pre-trained translation reward model MetricX (Juraska et al. 2024), which has demonstrated strong alignment with human judgments. Specifically, compute Kendall's coefficient between the ranking
20

---

Published as a conference paper at ICLR 2026
induced by semantic voting scores and the ranking based MetricX rewards. Table g presents the results. For reference, we also include the coefficient obtained from a random ranking. As shown, the semantic ooing-bbsed ranking exhibits substantially higher correlation with MetricX than random baseline.
K HYPERPARAMETER STUDY ON CNN/DAILYMAIL
In Section we examine how the hyperparameters HDBSCAN affect final performance. As that section focuses only translation tasks, provide additional results summarization. Specifically, Figure [4 shows the performance of Llama-IB on the CNN/DailyMail dataset. As can be seen, the conclusions drawn in Section 4.7 still hold: larger values of generally lead to better performance and greater robustness.
21


Reviews:
[{"review_id": "aprR1WxfnR", "reviewer": "Reviewer_nFpD", "summary": "The paper introduces Semantic Voting–based Self‑Improvement (SVSI) for tasks without verifiable answers. The method samples multiple answers, embeds them, and filters them down to the largest semantic cluster. Within this cluster, it ranks answers by their average cosine similarity to all other answers. The top-ranked (\"winner\") and bottom-ranked (\"loser\") answers are then used as a preference pair for DPO training. This approach uses a simple sentence encoder and clustering, avoiding expensive LLM-as-a-judge or entropy calculations to create preference pairs much more cheaply. Tests on translation and summarization tasks using six different models show SVSI consistently improves over the base model and generally performs as well as or better than self-judging or entropy-based methods. A control experiment confirms the signal is valid, as it degrades performance. The method is also tested as a dense reward for GRPO, where it's competitive but shows higher variance. Ablations show it's robust to different embedders and clustering tools.", "questions": "* How well does the \"top vote\" answer actually align with human judgment? A correlation study can be helpful to separate consensus from true quality.\n\n* Could the \"majority-error\" problem be fixed by retaining multiple clusters? Could \"hard negatives\" from outside the main cluster be used to improve DPO training?\n\n* How does this method scale to much larger models and longer, more complex outputs? How does it perform on non-English tasks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The mechanism is simple, general, and easy to explain, acting as a \"soft majority vote\" without needing a separate judge model.\n\n* Building preference pairs is computationally cheap, and the cost doesn't scale with the size of the base model, unlike LLM-as-a-judge.\n\n* The paper provides broad empirical evidence across multiple models, tasks, and metrics, including a useful \"flipped-signal\" control test.\n\n* Improvements generalize to out-of-distribution datasets, suggesting the model is learning a real capability, not just overfitting to the test set.", "weaknesses": "* The core assumption (consensus = correctness) can be a flaw. The method rewards conformity and could reinforce common errors while discarding correct but niche answers found in minority clusters.\n\n* Some baselines, like self-judging, perform poorly or collapse, making it unclear how much of SVSI's lead is due to its own merit versus the fragility of the alternatives.\n\n* The evaluation is limited to narrow tasks (translation to English, summarization). It hasn't been tested on long-form generation, conversation, safety, or creative tasks. The embeddings used are also English-centric.\n\n* The compute savings focus only on building the pairs, not on the sampling of answers, which is the most expensive part. True end-to-end time savings aren't clear.\n\n* When used as a dense reward for RL, the method seems less stable than discrete rewards.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oakxxoMhDu", "reviewer": "Reviewer_ikCz", "summary": "This paper focuses on addressing the limitations of existing LLM self-improvement methods for unverifiable open-ended tasks (e.g., machine translation, text summarization), such as high computational overhead and overconfidence bias in self-evaluation approaches (self-judging, entropy minimization) and the inapplicability of majority voting (relying on exact matching) to such tasks. It proposes the\nSemantic Voting-based Self-Improvement (SVSI)\nframework, a self-evaluation-free solution.\n\nSVSI draws inspiration from majority voting but relaxes hard exact matching to soft semantic similarity matching. The framework first enables an LLM to generate multiple candidate responses for a given input,then uses a lightweight sentence embedding model (e.g., SimCSE) to encode these responses into semantic vectors. After filtering outliers via density-based clustering (HDBSCAN) and retaining only the largest cluster to ensure meaningful semantic consensus, it calculates each response’s \"voting score\" as the average cosine similarity with other responses in the cluster. The highest and lowest scoring responses form preference pairs, which are used to fine-tune the LLM via Direct Preference Optimization (DPO).\n\nEmpirical evaluations across 6 models (Qwen series: 1.5B, 3B, 7B; Llama series: 1B, 3B, 8B) and 6 datasets (4 WMT24++ translation subsets, CNN/DailyMail and PubMed summarization datasets) show that SVSI consistently outperforms self-judging (SJ) and entropy minimization (EM) baselines in most cases, with computational overhead only 0.1%–5% of self-evaluation methods. Additional validations (e.g., flipped-SV experiments to confirm valid semantic signals, out-of-distribution tests to verify generalization) and ablation studies (on clustering and embedding models) further confirm SVSI’s effectiveness, stability, and adaptability.", "questions": "1.Given that the core logic of semantic voting (semantic clustering + similarity-based scoring) is relatively simple and lacks significant technical innovation, and the ablation study shows clustering is not universally useful (e.g., it does not harm Qwen-1.5B’s performance when removed), how does the paper justify the novelty of the proposed framework compared to existing semantic similarity-based or clustering-based methods in LLM self-improvement?\n\n2.In the related works section, the paper only critiques the limitations of existing methods (e.g., majority voting’s reliance on exact matching, self-evaluation’s high overhead) but fails to link these flaws to specific modules of semantic voting (e.g., lightweight sentence embedding, soft matching). Why does the paper not explicitly explain which module addresses which drawback of prior work, and how to verify the causal relationship between these modules and the solved problems?\n\n3.Figure 1 (the SVSI framework overview) references Section 3.2 (clustering-based filtering) before Section 3.1 (semantic voting), which contradicts the logical order of the Methodology chapter. What is the reason for this disordered section reference, and does it reflect a misunderstanding of the framework’s own workflow?\n\n4.Eq. 5 (the DPO loss formula) is directly adopted from Rafailov et al. (2023) without any modifications,\nyet it is included in the Methodology section. Since this equation is not an original contribution and is\nirrelevant to the core semantic voting mechanism, why does the paper choose to present it, and what\nvalue does it add to explaining the proposed method?\n\n5.Figure 4 (comparing semantic voting and flipped-SV) lacks a clear legend, making it impossible to\naccurately identify which curve represents \"SV\" and which represents \"flipped-SV,\" and unclear how\nthe axes correspond to lexical/semantic metrics. Why is such a critical result figure missing key\nannotation information, and how does this affect the credibility of the conclusion that \"flipped-SV\nunderperforms\"?\n\n6.The paper selects HDBSCAN as the clustering algorithm for filtering self-generated responses but\nprovides no experimental comparison (e.g., with k-means, DBSCAN) or theoretical justification (e.g.,\nalignment with semantic voting’s needs) for this choice. What evidence supports that HDBSCAN is\nmore suitable for semantic voting than other common clustering algorithms?\n\n7.HDBSCAN’s performance depends on sensitive parameters (minimum cluster size m and\nneighborhood threshold k), and the paper shows larger m generally improves performance. However,\nit does not explain whether these parameters need to be tuned for different tasks (e.g., translation vs.\nsummarization) or models. How should users determine the optimal m and k values when applying\nSVSI to new unverifiable tasks?\n\n8.For evaluation, the paper uses different metric pairs (BLEU/n-MQM for translation, ROUGE-L/BLEURT\nfor summarization) but provides no explanation for these choices. Why are these specific metrics\nselected instead of others (e.g., using BLEURT for translation or n-MQM for summarization), and how\nto ensure the selected metrics can fairly reflect the method’s performance across tasks?\n\n9.The method is designed for unverifiable open-ended tasks but only tested on translation and\nsummarization. No experiments are conducted to check if SVSI causes negative effects (e.g., reduced\naccuracy) in closed-ended tasks (e.g., arithmetic, multiple-choice). Why is the scope of experimental\nverification so narrow, and how to confirm the method’s safety when applied to mixed task scenarios?\n\n10.In Table 1, SVSI shows no significant advantage over baselines (e.g., on Llama-3B’s wmt24pp de task,\nEM’s BLEU score is 10.52 while SVSI’s is 8.71). The paper claims SVSI is \"effective,\" but why does it\nnot address the cases where it underperforms baselines, and how to prove its superiority in practical\napplication scenarios?\n\n11.Section 4.5 notes that when SVSI is adapted to the GRPO framework (SVSI-G), it exhibits obvious\nperformance fluctuations, which are more severe than EMPO’s stability. Since the paper emphasizes\nSVSI’s \"efficiency and robustness,\" why does it not solve the stability issue in alternative training\nparadigms, and what limitations does this reveal about the method’s adaptability?\n\n12.The ablation study (Section 4.6) shows clustering is unnecessary for high-capability models like\nQwen-1.5B (removing it slightly improves performance) and only beneficial for low-capability models\nlike Llama-1B. As a core component of SVSI, why does clustering lack universal necessity, and how to\nadjust the framework to avoid redundant steps for high-capability models?\n\n13.SVSI relies on sentence embedding models (e.g., SimCSE) to calculate semantic similarity. If the\nembedding model fails to capture task-specific semantics (e.g., medical terminology in PubMed\nsummarization), the pseudo-label quality declines. How does the paper plan to address the\ndependence on embedding model performance, and is there a way to reduce the method’s sensitivity\nto embedding model selection?\n\n14.Semantic similarity metrics cannot distinguish between \"semantically consistent but low-quality\" and\n\"semantically distinct but high-quality\" responses, leading to \"conformity bias\" (prioritizing majority\nconsensus over individual accuracy). Does the paper acknowledge this limitation, and what\nimprovements can be made to avoid discarding accurate but niche responses?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "Targeted Problem-Solving for Unverifiable Tasks:\nThe paper addresses a critical gap in LLM self-improvement—existing methods either rely on computationally expensive, bias-prone self-evaluation (e.g., self-judging, entropy minimization) or hard-matching majority voting (inapplicable to open-ended tasks like translation and summarization). Its proposed Semantic Voting-based Self-Improvement (SVSI) framework avoids self-evaluation entirely, enabling lightweight and effective optimization for unverifiable open-ended tasks .\n\nInnovative and Efficient Core Mechanism\n:\nInspired by majority voting but extended to \"soft semantic matching,\" SVSI uses lightweight sentence embedding models (e.g., SimCSE) to quantify semantic similarity, replacing rigid exact matching. This design drastically reduces computational overhead—its cost is only 0.1%–5% of self-evaluation baselines—while maintaining effectiveness, making it scalable even for larger models .\n\nPractical Engineering Value\n:\nThe framework is operationally transparent (detailed implementation in Appendix B) and requires no complex dependencies—self-generation uses standard sampling, embedding/clustering rely on lightweight tools, and DPO training uses standard hyperparameters. This makes SVSI easy to reproduce and deploy in real-world LLM optimization pipelines .", "weaknesses": "The proposed semantic voting approach relies on a straightforward pipeline: clustering self-generated responses using semantic similarity and calculating voting scores via average cosine similarity between sentence embeddings. This core logic lacks impactful technical novelty, as both semantic clustering and similarity-based ranking are well-established techniques in NLP. Notably, the ablation study confirms that clustering is not universally beneficial—for high-capability models like Qwen-1.5B, removing clustering either preserves performance or slightly improves it, which further underscores the framework’s simplicity and its limited innovative contribution to solving challenges in unverifiable open-ended tasks .\n\nWeak Connection in Related Works\n:\nThe related works section only critiques limitations of existing methods (e.g., majority voting’s reliance on exact matching, self-evaluation’s high computational overhead) but fails to link these flaws to specific components of the proposed semantic voting. It does not clarify how modules like lightweight sentence embedding or soft semantic matching directly address the drawbacks of prior approaches, weakening the rationale for the new method.\n\nDisordered Section Reference in Figures\n:\nFigure 1, which outlines the SVSI framework, references Section 3.2 (self-generation filtering via clustering) before Section 3.1 (semantic voting)—a misalignment with the logical order of the Methodology chapter, causing confusion about the workflow sequence .\n\nUnjustified Inclusion of Non-Original equation\n:\nEq. 5 (pertaining to DPO loss) is directly adopted from prior work (Rafailov et al., 2023) without modifications. The paper provides no reason for including this non-innovative equation in the Methodology section, as it is irrelevant to the core semantic voting mechanism.\n\nAmbiguous Figure 4\n:\nFigure 4 (comparing semantic voting and flipped-SV) lacks a clear legend. Key elements—such as labels for \"SV\" and \"flipped-SV\" or detailed axis annotations for lexical/semantic metrics—are missing, making it impossible to accurately interpret performance differences between the two settings .\n\nUnjustified Clustering Algorithm Selection\n:\nThe paper uses HDBSCAN for clustering self-generated responses but offers no experimental, theoretical, or literature-based justification for this choice. It does not compare HDBSCAN with other common clustering algorithms (e.g., DBSCAN, OPTICS) to demonstrate its suitability for semantic voting .\n\nSensitive and Opaque HDBSCAN Parameters\n:\nHDBSCAN’s performance depends on two key parameters (minimum cluster size m and neighborhood sample threshold k), which are shown to be sensitive to task and model variations (e.g., larger m improves performance for most models). However, the paper does not discuss whether these parameters require task-specific tuning or provide guidelines for adapting them to different unverifiable tasks, increasing practical application complexity .\n\nUnjustified Metric Choices\n:\nFor evaluation, the paper uses different metric-model pairs across tasks (e.g., BLEU/n-MQM for translation, ROUGE-L/BLEURT for summarization) but provides no explanation for these selections. It fails to justify why these metrics are optimally suited for measuring output quality in translation vs. summarization, weakening result validity .\n\nLimited Task and Domain Coverage\n:\nThe method is designed for unverifiable open-ended tasks but only tested on translation and summarization. No experiments check for potential negative effects in closed-ended tasks (e.g., arithmetic, multiple-choice), which is critical for assessing generalizability. The narrow task scope also makes it hard to confirm effectiveness across diverse unverifiable scenarios .\n\nNo Significant Advantage Over Baselines\n:\nIn Table 1 and Figure 2, SVSI performs on par with or slightly better than baselines (SJ, EM) but shows no substantial, consistent advantage. For example, on Llama-3B, EM outperforms SVSI on multiple translation datasets (e.g., wmt24pp de BLEU: 10.52 vs. 8.71), failing to demonstrate value beyond existing methods .\n\nStability Issues in GRPO Adaptation\n:\nAs noted in Section 4.5, when adapted to GRPO (SVSI-G), the method exhibits severe performance fluctuations across runs—far less stable than EMPO. This undermines confidence in its applicability beyond DPO training .\n\nUnproven Clustering Necessity\n:\nThe ablation study (Section 4.6) shows clustering is unnecessary for high-capability models like Qwen-1.5B (removing it slightly improves performance) and only beneficial for low-capability models like Llama-1B. This indicates the clustering component— a core part of SVSI—lacks universal necessity .", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sQ7fy0AAZD", "reviewer": "Reviewer_F69H", "summary": "This paper proposes Semantic Voting (SVSI), a lightweight and self-evaluation-free framework for improving large language models (LLMs) on unverifiable open-ended tasks such as translation and summarization. Unlike existing self-improvement methods (e.g., self-judging or entropy minimization), which rely on computationally costly and biased self-evaluation, SVSI replaces hard exact matching with soft semantic matching using sentence embeddings. The method clusters self-generated responses, applies semantic similarity-based voting, and uses the resulting pseudo-preference pairs for Direct Preference Optimization (DPO) training. Experiments across six models (Llama and Qwen series) and multiple datasets show that SVSI achieves comparable or superior performance to baselines while being 100× more computationally efficient.", "questions": "- How sensitive is SVSI to the choice of sentence embedding model? Would using multilingual or domain-specific encoders improve results?\n\n- Could the method scale to even larger sampling sizes (e.g., 256 responses) without diminishing returns or excessive memory cost?\n\n- Did you consider using model-based similarity scoring (e.g., cross-encoder) instead of embedding-based cosine similarity for higher fidelity?\n\n- In the flipped-SV experiments, could the residual improvements over base models be due to pretraining biases or unintentional preference alignment?\n\n- How would SVSI perform in reasoning or dialogue-generation tasks where “semantic similarity” may not correlate with quality?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Introduces a new paradigm (semantic voting) that generalizes majority voting to open-ended tasks through soft semantic matching.\n\n- Demonstrates a drastic reduction in computational cost (as low as 0.1–5% of self-evaluation methods) without sacrificing performance.\n\n- Extensive evaluation across 6 models and multiple tasks with consistent gains in BLEU, ROUGE-L, and BLEURT.\n\n- Includes flipped-pair tests, ablation studies, and hyperparameter analysis that strengthen methodological credibility.\n\n- Algorithmic clarity and inclusion of implementation details in the appendix enhance replicability.", "weaknesses": "- The semantic similarity metric depends heavily on the choice of the sentence embedding model (SimCSE). Cross-model generalization is not deeply analyzed.\n\n- Evaluations focus mainly on translation and summarization. It remains unclear how SVSI would perform on more subjective tasks (e.g., story writing or reasoning).\n\n- Results rely solely on automatic metrics; human judgments would provide stronger validation of semantic quality improvements.\n\n- As shown in Section 4.5, reward-based training variants exhibit performance fluctuation. The paper could provide further analysis on stabilization strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "US69Amw2qP", "reviewer": "Reviewer_jSCx", "summary": "This paper introduces Semantic Voting (SVSI), a novel self-improvement framework for LLMs on open-ended, unverifiable tasks. The core idea is to extend majority voting to the semantic space by generating multiple candidate responses, clustering them to find a semantic consensus, and then using the average semantic similarity as a score to construct preference pairs for DPO training. The method aims to be a computationally efficient and self-evaluation-free alternative to existing approaches.", "questions": "1. How does the method impact output diversity? Have you measured whether it causes a Matthew effect, leading to a less diverse distribution?\n\n2. How can the method be adapted for long-text tasks where a single embedding is insufficient?\n\n3. Were other unsupervised methods considered for comparison? For the tasks and models chosen in this paper, Supervised Fine-Tuning (SFT) would likely outperform SVSI with better cost-efficiency. Could you elaborate on the necessity of using SVSI in this context?\n\n4. Can you explain the observed signal instability (e.g., with GRPO) and the method's sensitivity to its components, like the choice of embedding model?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. SVSI offers a practical and computationally lightweight alternative to expensive self-evaluation methods (like self-judging) for unverifiable open-ended tasks.\n\n2. The methodology is well-structured and clearly explained.", "weaknesses": "1. For unverifiable, open-ended tasks, diversity is a crucial metric. However, the proposed method may stifle diversity and creativity, as this unsupervised approach could lead to a severe Matthew effect. An answer that is creative, novel, or stylistically unique (yet still correct) is likely to be filtered out or receive a low voting score simply because its semantics differ from the 'mediocre' majority.\n\n2. Its reliance on sentence embeddings makes its generalizability to long-text tasks questionable, where a single vector cannot capture full semantic complexity.\n\n3. The experimental comparison is also limited, omitting more baselines and modern evaluation metrics like LLM-as-a-judge.\n\n4. The cost analysis is potentially misleading; while cheaper than self-evaluation, the method's absolute cost is significantly higher than Supervised Fine-Tuning (SFT), yet no performance comparison is provided to justify this.\n\n5. The pseudo-reward signal appears inherently unstable, as shown by its performance fluctuations with GRPO, suggesting its success may heavily depend on the noise robustness of the DPO framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)