SpeechJudge: Towards Human-Level Judgment for Speech Naturalness
Authors: Xueyao Zhang
Chaoren Wang
Huan Liao
Ziniu Li
Yuancheng Wang
Li Wang
Dongya Jia
Yuanzhe Chen
Xiulin LI
Zhuo Chen
Zhizheng Wu
Venue: ICLR 2026 Poster (2026)
Keywords: speech naturalness
human dataset
RLHF
generative reward model
AudioLLM
Primary area: datasets and benchmarks
TL;DR: We propose SpeechJudge, a suite centered on speech naturalness, which includes a human preference dataset, an evaluation benchmark, and a generative reward model.

Abstract: Aligning large generative models with human feedback is a critical challenge. In speech synthesis, this is particularly pronounced due to the lack of a large-scale human preference dataset, which hinders the development of models that truly align with human perception. To address this, we introduce ***SpeechJudge***, a comprehensive suite comprising a dataset, a benchmark, and a reward model centered on naturalness—one of the most fundamental subjective metrics for speech synthesis. First, we present ***SpeechJudge-Data***, a large-scale human feedback corpus of 99k speech pairs. The dataset is constructed using a diverse set of advanced zero-shot text-to-speech (TTS) models across diverse speech styles and multiple languages, with human annotations for both intelligibility and naturalness preference. From this, we establish ***SpeechJudge-Eval***, a challenging benchmark for speech naturalness judgment. Our evaluation reveals that existing metrics and AudioLLMs struggle with this task; the best-performing model, Gemini-2.5-Flash, achieves less than 70% agreement with human judgment, highlighting a significant gap for improvement. To bridge this gap, we develop ***SpeechJudge-GRM***, a generative reward model (GRM) based on Qwen2.5-Omni-7B. It is trained on SpeechJudge-Data via a two-stage post-training process: Supervised Fine-Tuning (SFT) with Chain-of-Thought rationales followed by Reinforcement Learning (RL) with GRPO on challenging cases. On the SpeechJudge-Eval benchmark, the proposed SpeechJudge-GRM demonstrates superior performance, achieving 77.2% accuracy (and 79.4% after inference-time scaling @10) compared to a classic Bradley-Terry reward model (72.7%). Furthermore, SpeechJudge-GRM can be also employed as a reward function during the post-training of speech generation models to facilitate their alignment with human preferences.

Published as a conference paper at ICLR 2026
SPEECHJUDGE: TOWARDS HUMAN-LEVEL JUDGMENT FOR SPEECH NATURALNESS
Xueyao Zhang' Chaoren Wang
Huan Liao' Ziniu Li'
Yuancheng Wang'
Li Wang' Dongya Jia² Yuanzhe Chen
Xiulin Li
Zhuo Chen
Zhizheng Wul.4.6,6r
The Chinese University of Hong Kong, Shenzhen
2ByteDance Seed
*DataBaker Technology "Amphion Technology Co., Ltd
Shenzhen Loop Area Institute
City University of Macau
ABSTRACT
Aligning large generative models with human feedback is a critical challenge. In speech synthesis, this is particularly pronounced due to the lack of large-scale human preference dataset, which hinders the development of models that truly align with human perception. To address this, we introduce SpeechJudge, a com- prehensive suite comprising dataset, benchmark, and a reward model centered on naturalness one of the most fundamental subjective metrics for speech syn- thesis. First, present Speech.Judge-Der of 99K speech pairs. The dataset is constructed using a diverse set advanced zero-shot text-to-speech (TTS) models across diverse speech styles and multiple languages, with human annotations for both intelligibility and naturalness pref- erence. From this, we establish JudeehJJuge--Eval speech naturalness judgment. Our evaluation reveals that existing metrics and Au- dioLLMs struggle with this task; the pest-performing model, Gemini-2.5- 2.5-Fals. achieves less than 70% agreement with human judgment, highlighting a signifi- cant gap for improvement. To bridge this gap, we develop Jpeech.Judge-GRM, a generative reward model (GRM) based on Qwen2.5-0mni-77 SpeechJudge-Data via a two-stage post-training process: Supervised Fine- Tunnng rationales followed by Reinforcement Learning
large-scale human feedback corpus
challenging benchmark for
It is trained on
(SFT) with Chain-of-Thouur (RL) with GRPO on challenging cases. the SpeechJudge- benchmark, the proposed SpeechJudge- GRM demonstrates superior performance, achieving 77.2 77.2% accuracy (and 79.4% after inference-time scaling @ 10) compared to a clas- sic Bradley-Terry reward model (72.7%). Furthermore, SpeechJudge- also employed as a reward function during the post-training of speech generation models facilitate their alignment with human preferences.
can be
INTRODUCTION
The collection and integration of human feedback corpora for model alignment has become a critical
stage in the development of modern large-scale generative models, proving indispensable in domains such as text (Stiennon al., 2020) Ouyang et al., [022 Bai al. [0022), image (Xu et al. 2023]
Kirstain et al. (023), and video generation (Xu et al. 2024 Liu et al. 2025b).
In the field speech synthesis, naturalness has long been a cornerstone subjective metric for qual- ity assessment (Ju et al. 2024 Anastassiou et al. 2024 Du et al. 2024a Xu et al. 2025 KemiTeam indicators of performance (Taylor) Tan 2023). ). Prior research has explored automated speech assessment through MÓS pre- dictors (Sacki et al. 2022; Huang et al.) 2024) and constructed the human feedback corpora for specific attributes like the low-level acoustic quality (Wang et al.) 2025c). However, large-scale human feedback corpus centered on the holistic quality of naturalness and a corresponding reward model trained capture these preferences-rentte remainss notably underexplored area. To fill this void, this paper focuses on the dimension of speech naturalness and present a three-part contribution:
ct al. 2025), representing one of the most general-purper
2009
"Equal Contribution.
1 "orresponning author.

---

Published as a conference paper at ICLR 2026
1. A Large-scale Human Feedback Dataset: Speech.Judge-Date We recruit human annotators to provide feedback on synthesized speeches, with a focus on assessing two fundamental speech as- pects: intelligibiliity and naturalness. For data synthesis, we employ diverse set of advanced, open- source zero-shot TTS models with varying architectures (such as Cosy Voice2 (Du et al.) 2024b),
Ints (Zhang et al.) (025b), F5-TTS (Chen et al. 2025b), and MaskGCT (Wang al. 2 to produce the compared speech pairs. We prepare speech references in both regular and expressive styles, construct multilingual target texts, and cover both monolingual and cross-lingual synthesis scenarios to ensure data diversity (Section 3.1 We instruct human annotators to perform two tasks based on speech pair (Figure []: (a) pointwise annotation of text accuracy to assess intelligibility, and (b) pairwise preference annotation to judge relative speech naturalness. This extensive effort, involving 69 labelers over two months, results in 99K annotated pairs, with each pair receiving an average of 2.49 annotations from different labelers. We believe the SpeechJudge-Date can serve as a valuable corpus for alignment research in speech synthesis (e.g.. DPO alignment (Rafailov et al., 2023) or reward modeling (Stiennon et al. 2020; Ouyang et al. 2022 Bai al. 2022) in Section
2. An Evaluation Benchmark for Speech Naturalness Judgment: Speech.Judee-Eval. We de- sign dedicated evaluation benchmark for the task of speech naturalness judgment. The task is structured as follows: given a target text and two corresponding speech samples, a model needs to judge which one is more natural. To construct the evaluation set, we select subset from the SpeechJuuge-Data where human annotators demonstrated high inter-annotator agreement, ensuring a high-quality ground truth. We assess the naturalness judgment capabilities of a wide range of metrics and models, including Word Error Rate (WER) (Radford et al.] 2023 Gao et al. 2023). Fréchet Audio Distance (FAD) (Kilgour et al.
2019). MOS predictors (Sacki et al. 2022
Reddy
et al. 2022, Tjandra et al. 2025). Deepfake Detectors (Jng et al.) 2022; Wang et al.
2025b). and
LLss (Xu et al.] | KimiTeam et al.) 2025| Xiaomi| 2025 Comanici et al.
2025 Hurst
et al. 2024). Our eva evaluations reveal that even the most capable model-specifiialll Gemini-2.5- less than 70% agreement with human preferences. This finding highlights a significant performance gap and underscores the substantial room for research and improvement in automated speech naturalness judgment.
Flash (Comanici et al. 2025) in our experiments-acciivee
3. A Generative Reward Model for Speech Naturalness: Speech To develop reward model that more effectively captures human preferences, we develop SpeechJudge- GRM, a generative reward model (GRM) (Zhang et al. 2025a Liu et (025c) trained on the SpeechJudge- Data. Specifically, we base our model on post-training process. During the first stage, we perform Supervised Fine-Tuning (SFT) as the "cold start" to improve the model's s instruction-following and rationale-based reasoning capabilities. To achieve this, we leverage Gemini-2.5-Flast (Comanici et al. 2025) to generate Chain-of- Chain-of-Thoughle Thought (CoT) data for speech naturalness judgment task. In the second stage, we focus on more challenging cases of SpeechJudge- Data, which we define as instances where Gemini-2.5-Flaa fails to make the correct judgment. Treating the human-annotate
al. and design a two-stage
labels as the verifiable reward (DeepSeek-A) et al.) Liu et al.) (022cc), we apply the GRPO-based Reinforcement Learning (RL) stage (Shao et al., Our experiments demonstrate that when trained on the same data, SpeechJudge-GRM sig- nificantly outperformed the classic Bradley-Terry reward model (BTRM) (Bradley & Terry 1952] Rafailov et al.) 2023), achieving higher accuracy in predicting human preferences (77.2% for SpeechJudge-Gre GRM vs. 72.7% for SpeechJudge- BTRM, Table Besides, SpeechJudge-GRe also supports inference-time scaling and offers explainability through its CoT outputs. Furthermore, SpeechJudge-GGRe can also be employed as an objective naturalness metric for sample selection (Figure 5) or as a reward function in RL algorithms enhance the quality of existing speech gener- ation models (Figure
2025
2024
We release all resources from this study at https//gthhllcooommmmmmlllllllln Audio samples are available at http:///ppeecuuuugeeeebbbbbbebee
2 RELATED WORK
Human Alignment for Speech Generation Aligning generative models with human feedback has proven crucial, process also known as RLHF in LLMs (Ouyang et al.) 2202 Bai et al. 2022 In the vision domain, many similar human preference datasets exist, such as Pick-a-Pic (Kirstain
et al. 2023) ImageReward (Xu et al. 2023), and Rewed (Liu et al. 2025b). The speech
2

---

Published as a conference paper at ICLR 2026
Speech Pair (with Target Text)
User Interface of the Human Annotation System
Speech
Speech
With MHN Hijy
Ie Bad In Eii ja de lin lat las las Las
that momonnt (6 stars whippering the moon smilling, looked
that moment, (6 the staro whrloeeing the rico smillingg llooeed up heart fileed with end fantaaiees JB BRAKEE exploring the boundless universo, Maaybe that distartt place. 8888 22 (8 2eiir milocless waiting
skyy, heart filod with endless farnasies, 18 22 R, ougroring the boundieess
universee Maybe that distant place, #4 tu---t S2aten miloddess weiking for
eiseover. 88-88-11 ( story, f8 (0-48--7 10--0-------- St: PIS- longing, longing anioul
discovos 18-2812 8550-9 story, 90--ip se FRt64 longing, longing for arrivd.
DO 10/000 20:25.00
DOlCUCo (0028
Intelligibility Annotation (pointwise)
error? linsertion,
omission, or mispromunciation)
Speech
C
Hes Emor
No Error
Speech Error
Which speech sounds natural?
(b) Naturalness Annotation (painwise)
Figure I: SpeechJudge- Data consists of speech pairs (with corresponding text) synthesized by mul- tiple zero-shot TTS models. For each pair, human annotators need to perform (a) pointwise anno- tation of text accuracy to assess intelligibility, and (b) a pairwise preference annotation to judge the relative speech naturalness.
synthesis field, pioneering efforts to construct human corpora involved MOS datasets (Saeki et al., 2022 Huang et al. 2024). However, these datasets often did not use advanced TTS models for data generation, provided only the pointwise labels rather than the direct pairwise human preference, and were limited in scale. More recently, efforts have focused on building human feedback cor- pora centered specific speech attributes, such as low-level acoustic quality (Wang et al. 2025c), capabilities spoken dialogue sys- tems (Ji et al. [2255] Ge et (2025). Despite this progress, a large-scale human feedback corpus built specifically around naturalness one of the most general-purpor and fundamental metrics for
intelligibility (Zhang et al., allaa1122220 (025b), or the instructioon-ollowwe
speech synthesis [Taylor 2009
Tan 2023
has remained critical missing piece.
AudioLLM as Judge Using LLMS as automated quality evaluators is a prominent topic in the textual LLM field, popularized by the as--uus---j paradigm (Zheng et al., 2023). This idea has recently been extended to the audio domain. A concurrent work, AudioJudge (Manakul et al., 2025), evaluates the capabilities and limitations of using AudioL..Ms for speech quality assessment and paralinguistic understanding via prompt engineering. Furthermore, many studies have focused on fine-tuning AudioLLMs to better expose their understanding capabilities for specific tasks, such as discriminating the human-likeness likeness of audio (Wang et al.) (0255)), modeling low-level acoustic qualities (Chen et al. 2025a Wang et al. 2025c)), unifying multiple speech quality evaluation tasks into single AudioLLM (Wang al. 2025a). and enhancing the assessment of cnttrucctoo-followiiii in spoken dialogue systems @Ji et al. 2025 Ge et al.) 2025). However, how to improve the ability of AudioL_L.Ms to understand and judge speech naturalness, and how their quality-assessment capabilities as a reward to improve the post-training of speech generation models themselves, remain significantly underexplored.
3 SPEECHJUDGE-DAAE
Our work grounded SpeechJJudge-DDe a large-scale human feedback corpus for assessing the intelligibility and naturalness of synthesized speech. Formally, we aim to construct dataset D = {(t, 01, a2)). where each triplet comprises a pair of synthesized speech samples (a1, a2) and the corresponding target text f. We instruct annotators to provide pointwise intelligibility and pairwise naturalness preference annotations based on D (Figure
3.1 DATASET CONSTRUCTION
We employ a diverse set of recent advanced zero-shot TTS models to prepare the dataset D. For- mally, for each sample (t, a1 a2). we denote the synthesized speech a, as being produced by the model Mus. i.e., ag Mus(are) 1), where aref is the reference speech.
3

---

Published as a conference paper at ICLR 2026
17.9%
16.5%
12.8%
18.2%
ARS
Emile-
CosyVoice2
Genahin Impact
mixed
35.1%
15.5%
Conytoice2-NHTP FS-TTS
18.6%
KeSpeach
In-house Whisper
Inte-INTP
HaskGCT
69.9%
ParaSpecchCups
L2-Arctic
18.4%
zh2mixed zh2ah
14.5%
25.8%
(a) TTS Models.
(b) Speech References.
(c) Language Settings.
Figure 2: Distribution of SpeechJudge-Diie
Model Selection For Mass. we select the following six models of three architectures to en- rich the distribution of the synthetic data (Figure 2a (1) AR-based: ARS (Wang et al. (025e).
CosyVoicet (Du et al., (2024)), Cosy 2oiC (Zhang et al., 2025b), and Ints-INTP (Zhang (et al. 2025b). The latter two are released by Zhang et al. (2025b) as inteliggbbllty.ennnneee models.
(2) FM-based: F5-TTS. (3) MGM-based: MaskGCT (Wang al. 2025e).
Prompt Construction To build diverse prompts (aref, t) for TTS, for aref. we adopt both regular and expressive speech samples. The regular samples are randomly selected from the Emilia- Emilia-Large dataset (He et al.) 2025). The expressive samples are sourced from corpora rich in paralinguistics, including the emotional corpora: ParaSpeechCaps (Deb et al. 2024), the accented corpora: L2- Arctic (Zhao et al.) 2018) and KeSpeech (Tang et al. 2021), the whisper samples from an in-house corpus, and the character voices from video games Genshin Impact (simon3000) 2023). We display the detailed distribution of speech references in Figure 2b
The target text paired with each aref is constructed as follows: For regular Oref samples, we randomly sample transcriptions from the Emilia-Large dataset (He et al. 2025). These are then refined using DeepSeek- V3 (DeepSeek-AV) et al. 2024) to correct typos and normalize punctuations. For expressive f samples, we instruct DeepSeek- V3 to generate several scripts in different writing styles, tailored the topic of aref (see Appendix B.T for more details). The languages of the target texts included Chinese (zh), English (en), and Chinese-Engllsh code-switching (mixed). For the combinations (areft 1), we include both monolingual settings (en2en and zh2zh) and cross-lingual settings (zh2en, en2zh, zh2mixed, and en2mixed), where zh2en denotes Chinese aref with English t, and similarly for others. The distribution of the language settings of (Aref: t) is shown in Figure 2c
Speech Pair Construction To ensure the diversity of the (a1, ag) pairs being compared, we fol- low Zhang et al. (2025b) and adopt both intra-model (i.e.,a and 02 being generated by the same model) and inter-model pairs (i.e., d1 and d2 being generated by the different models). The distri- bution of the speech pair is shown in Figure
3.2 HUMAN ANNOTATION
Given a sample (1, a1, (2), human annotators instructed perform both pointwise intelligibil- ity and pairwise naturalness annotations (Figure [p. For intelligibility, annotators perform binary classification determine whether the speech (a, and a2) accurately reads the text without any content insertion, omission, or mispronunciation. For naturalness, they perform a five-scale Com- parative Mean Opinion Score (CMOS) annotation to determine which of the two audio clips (a) or a2) sounds more natural and human-like
We recruited professional annotators from a specialized data annotation firm in China and provided them with training for speech naturalness judgement. All annotators assigned to Chinese data were native speakers. For the English and code-switching datasets, annotators were required to have a proficiency level equivalent to at least CET-6. All personnel underwent standardized training based on a detailed annotation manual. Initially, we conducted a pilot study among researchers to refine the guidelines for clarity and unambiguity. To ensure annotation quality, each sample (t, a1 (2) was independently annotated by two individuals. A third annotator was introduced if any disagreements. The detailed annotation guidelines provided in Appendix C
Statistics
We recruit 69 annotators and conduct annotations over two months. The resulting con- structed dataset D. which denote as SpeechJudge- (raw), contains 99K (1, a1, a2) samples,
4

---

Published as a conference paper at ICLR 2026
with each sample receiving average of 2.49 annotations from different labelers. The market value of this annotation scale is estimated at over 500K RMB (about 70K USD). Based on the raw dataset, also construct several subsets for analysis and reward model training. We provide detailed de- scriptions of each subset and its applications in the following sections and in Appendix B.2
Human Agreement Analysis We analyze the human annotations for naturalness in this section; discussions regarding intelligibility are provided in Appendix C333 For naturalness annotations, we evaluate the inter-annotator agreement across our constructed dataset. To simplify the analysis, given the sample (t, 01, (2), we transform the five-scale naturalness scale (CMOS) into ternary classification system: either a1 better, 02 is better, or their quality is Tie. Based on this simplified classification, we categorize the annotation results into four distinct levels of agreement (1) Full Agreement (FA): A consensus is reached among all annotators, with all ratings pointing to the same outcome (e.g.. "2A", "3A", "2B", "3B"). We use "2A" to indicate that two annotators both rated a as better, while "3B" denotes three annotators all rating dg as better. (2) Weak Agreement (WA): This level captures cases where two annotators agree a specific polarity, while the third annotator marks a Tie (e.g., "2A+IT", "2B+IT""). We also include the "2T+IA" and "2T+1B" cases in this level. (3) Weak Disagreement (WD): This occurs when two annotators' ratings share the same polarity, but the third's rating is the opposite (e.g., "2A+B", "2B+A"). (4) Full Disagreement (FD): This represents complete lack of consensus, where all three annotators provide different classifications, denoted as "IA+1B+IT".
Regular
54.5%
Expressive
44.3%
23.3%
15.8%
All
51.4%
17.2%
19.1%
12.3%
0%
20%
40%
60%
80%
100%
Figure 3: Distribution of SpeechJudge- Data on different levels of human agreement.
In Figure 3] we demonstrate the distribution of these human agreement levels for the SpeechJudge- Data and its two subsets, regular and expressive (which are defined by their speech references). The figure shows that about 70% of the entire dataset falls into the Full Agreement (51.5%) or Weak Agreement (17.2%) levels. Furthermore, we observe that the expressive subset has lower agreement level than the regular subset, which suggests that human evaluation of expressive speech generation is inherently more challenging problem. Besides this sample-level agreement analysis. we also analyze the reliability of individual annotators, and we will discuss this in the Appendix
SPEECHJJDGE--VAL
To evaluate speech naturalness, existing studies typically organize their own listening tests, which often have inconsistent settings across different papers (Ju et al., 2024, Anastassiou et al., 2024 Du et al., 2024a Xu et al., 2025: KimiTTem et al., 2025). Alternatively, previous researchers use proxy MOS predictors, such as UTMOS (Saeki et al., 2022), as an objective metric. However, it remains an underexplored problem whether these metrics can accurately judge the naturalness of more advanced speech generation models (Wang al. 2025e; Du et al., 2024b) [Chn et al. 2025b. Zhang et al., (025b) and align with human preferences. Motivated by this, construct benchmark, Speech Judge-hudeeeeeeeo Eval, specifically for the speech naturalness judgment task.
4.1 TASK DESCRIPTION
Task Formulation We formulate the naturalness judgment task a pairwise comparison, specif- ically a win-or-lose binary classification task: Given a target text and a corresponding audio pair (a1,as) model needs to determine which audio has better naturalness. This results in binary choice: either 0j is better or 02 is better. We use the human answer as the ground truth, and use
Note: Each sample of SpeechJudge- is independently annotated by a minimum of two and maximum of three annotators (Appendix C
5

---

Published as a conference paper at ICLR 2026
Table 1: Protocols of different mod- els for naturalness judgment.
Table 2: Accuracy of speech naturalness judgment across
different models on SpeechJudge-Eval.
Type
Protocol
Model
Regular
Expressive
Total
WER J. Naturalness f
Objective Metrics
Objective Metries
SIM 1. Naturalness 1
WER
59.3
57.0
57.9
FAD 4, Naturalness f
SIM
47.5
42.5
44.5
MOS Predictors
FAD
50.3
47.5
48.6
MOS 1, Naturalness
MOS Predictor
Deepfake
DNSMOS
61.0
55.8
57.9
Fake J, Naturalness f
Detectors
UTMOS
54.0
53.5
53.7
AudioLLMs
Score Naturalness
Content Enjoyment (CE)
69.3
55.2
60.8
Content Usefulness (CU)
61.3
54.7
57.3
Prompts of A.d
Production Complexity (PC)
39.3
48.7
44.9
We comparing the naturalness of two models' ourpuis. The models need to speak the target text accurately and naturally.
Production Quality (PQ)
61.3
54.3
57.1
Deepfake Detectors
AASIST ADV
40.5
50.8
46.7
35.3
40.3
38.3
Target text: (1). Output A: {ar), Out-
put {az]- Analyze
AudioLLMs (Open-source)
above, and score them with num- ber from 1 to 10. Note:
Mul---Mulimodal
54,8
58.5
57.0
Qwen2.5-0mmi-77
62.0
59.7
60.6
Please evaluate the naturalness of both andio ourputs based on the following criteria: Prosody and Intonation. Pacing and Rhythm. Articulation and Clarity. and Overall Naturalness.
Ball-Auddo----nstttee
65.5
68.0
67.0
Gemma-3n- 3n-E4B-it
49.0
47.7
48.2
Voxtral-Mini-35t Min--38--50-7
60.0
53.3
56.0
MiDashengL.M
58.8
63.5
61.6
HiMo-Audion 7B Instruct
61.3
49.3
54.1
After conducting a detailed unal- ysis of each criterion, using the following output template to high- light your conclusion: Output A:
AudioLLMs (Closed-source)
Gemini-2.5-Flan 2.5min
73.5
66.2
69.
73.0
62.2
66.5
X, Output
GPT-4o mini Audio
56.3
46.7
50.5
GPT-4o Audio
71.5
64.7
67.4
We instruct MusioLLMs using two modes of prompt: plain and COT. The text in blue is only employed during the CoT mode.
We use the protocols of Table to establish judgment rules for dif- ferent models. The results of AudioL.MS here are obtained using the plain prompt of Table
Accuracy to measure the judgment performance of a model M on the evaluation set D:
[2]
Accuracy
I(um VH),
[D]
where (D) is the total number of samples in the evaluation set, V.M and y/ represent the answers of the model M and human for the sample d, respectively. II is the indicator function.
Evaluation Data We sample subset of the SpeechJudge- to create the evaluation set for SpeechJuuge-Eval. Specifically, we first select subset that contains only preference data (i.e., we filter samples with the "Tie" annotation), and then choose only those with full-agreement-level (FA) samples to ensure high-quality ground truth. We perform sampling from both the regular and expressive subsets of SpeechJudge- en. and mixed) within each subset. The final SpeechJudge-Ewv dataset consists of 1,000 samples. The construction details of SpeechJudge Eval and its distribution can be found in Appendix B.2
and proportionally cover the three target text languages (zh,
4.2 BENCHMARK FOR DIFFERENT MODELS
We test the naturalness judgment capability of various models based on SpeechJudge- Eval. We consider four different categories models, whose evaluation protocols are shown in Table 1. Objective metrics, such as WER (Radford et al. 2023 Gao et al.) 2023). SIM (Chen et al.)
2022), and FAD Kilgour et al., 2019) in audio generation tasks. We assume that a better value of these metrics (e.g., lower for WER and FAD; higher for SIM) indicates better naturalness.
2. MOS Predictors, including DNSMOS Reddy al.) 2022). UTMOS (Sacki al. 2022), and aesthetics andiobox-aesthettis
predictors from audiobox
(CE, PC, and PQ) (T)andra et al. 2025).
We assume
that higher MOS score corresponds better naturalness.

---

Published as a conference paper at ICLR 2026
Chain-of-Thought Prompt
Rationale-based Output
SFT Dataset (CoT Peorepo Ouqpsaa
wosk ung only oft sourth
(K0)
paliy comaater catiiii the
Preference Parse
Bi mamter from in
A Audi 8 Q
unrum booton securii
leeeratie: proudd and aecodio
Human Verilication
Chitte
sideeltio and tau do
RL Dataset (CoT Proceptto
Oupuric I. Dersel 3.X.
Cercedre Compon 16 Dupuur Ii3
Gemini-2.5-Flesh
SFT & RL Datasels
Human Preference as Verifiahle Reward
(b) RLVR
RL Dataset
Hout matiar
Pours calk
Horth unanciattall
(CuT Prompe))
Ourgutt: at
Ouppar
Dupert::
Ougeutt
Policy Model
Rollonts
Artonaaio Clarita...
Artanlados Currgi..
Arlidadie Carigo
GRPO
Packc sauthee Chirli.....
Fakiig lookker indeeedessaa Currer....
Faiiy Arlecedii Carrity
Packi Muster Cherly....
Human Preference
Condiatie OuDertA Oup
Canturie: To
Cendaiex 0uper
A
Audio A
Andia B Q
A 1. Audio e
Audio A O
Asdo 4
Audio A Audoo B 6
Audio
Accuracy Reward:
Accuracy Reward: -1
Accuracy Reward:
Accuracy Rewards
Figure 4: SpeechJJude- (a) We employ Gemini-2.5-Flash as a teacher model to generate CoT rationales for SpeechJudge- Data. We use the samples where Gemini-2.55-llasee 's preference aligns with human as the SFT dataset, while the remaining samples are reserved for the RL stage. (b) We treat the human preference as verifialle reward to train the GRM with GRPO.
3. Deepfake detectors, which are typically pre-trained on a binary classification task to predict whether an audio is fake not (Jung et al. 2022 Wang et al. 2025b). We assume that an audio with a lower fake probability should have better naturalness.
4. AudioLLMs, which are employed to test their speech naturalness understanding capabilities in
zero-shof m We include the open-source Phi-4-Multimodal (Abouelenin et al. 2025).
Owen2.55-mmi (Xu et al. 2025), Kimi-Audio (KimiTeam et al., 2025). Gemma-3n (Kamath
et al.
2025 Voxtral (Liu al. (025a). MiDashengLM (Dinkel al. 2025), Mimo-Audio (Xi-
aomi 2025). and the closed-source Gemini-2.5 (Comanici et al. 2025) and GPT-4o (Hurst et al.)
2024
We use the plain prompt of Table to instruct the model to pairwise score the naturalness of two audios. We use their grading to determine the naturalness preference.
The performance of different models on SpeechJudge-Evee is presented in Table 2 A key observa- tion that speech naturalness judgment is highly challenging task. The best-performing model, Gemini-2.5-Flash, still only achieves less than 70% agreement with human preferences. When com- paring different models, we find that: (1) common objective metrics and MOS predictors show only weak correlation with human preferences, often achieving less than 60% accuracy and sometimes performing at the level of a random guess (around 50%). (2) While deepfake detectors are highly effective at distinguishing between machine machine-generated generetegeerddeeeee and human-recorded speech (Wang et al.) 20256, Jung et al., 2022), their ability to do so is not well-aligned with the naturalness objective when comparing two generated samples. (3) AudioLLMs demonstrate significant potential for this task. While some models, such as Gemma-3n and GPT-4o mini Audio, perform at a chance level, a number of others achieve accuracy exceeding 60%. This promising performance motivates to further leverage these AudioLLMs for the design of a reward model for speech naturalness.
5 SPEECHJUDGE-GRM
Based on the proposed SpeechJudge- we further explore how to train a reward model capable of accurately capturing human preferences. Specifically, we propose Jheech.Jugg-GRRM, where we leverage the inherent audio understanding capabilities of AudioL.LMs (specifically, Qwen2.5- Omni-7B (Xu al. 2025)) to clicit their speech naturalness judgment capability. Compared to the classic BTRM (Bradley & Terry 1952), the key strengths of GRM are its ability to enable Chain- of-Thought (CoT) reasoning and its support for test-time computation via majority voting, which ultimately leads to improved preference judgment performance (Zhang et al. 2025a).
We assume that our adopted Audiol L.M have not been directly trained on the speech naturalness judgment task. Their performance this benehmark is therefore considered a zero-shot capability.
7

---

Published as a conference paper at ICLR 2026
Table 3: Accuracy of speech naturalness judgment of SpeechJudge-GRM.
Win
Speschjudge
32.59%
Speechjudge- GRM
30.20%
25.30%
43.00%
Model
Regular
Expressive
Total
0%
20%
40%
60R5
50%
100%
Qwen2.5-Omiit
62.0
59.7
60.6
Gemini-2.5-Flash
73.5
66.2
69.1
Figure 5:
Subjective evaluation of us- ing SpeechJudge-GRN for high-naturalness sample selection. Human subjects com-
SpecchJudge-BIeg -TTr
77.5
69.5
72.7
SpeechJudge-RRM. (SFT)
77.8
73.7
75.3
w/ Voting @ 10
77.4
77.6
77.6
SpeechJudge-GRM (SFT-RL)
79.0
76.0
77.2
pare a best-of-100 output of Qwen2.5-Omn
w/ Voting@e
80.5
78.7
79.4
7B (Talker), chosen by either SpeechJudge-
Our ovaluation is conducted on SpeechJodge-Eve (Like Tablo 2 Voting@ 10; For each prompt, the GRM gencrates 10 octputs, and we use the majority voting from these 10 octputs as the final result.
BTRM or SpeechJudge-GRee against a ran- domly output.
5.1 METHODOLOGY
We develop SpeechJudge- GRM based Qwen2.5-Ommi-- (Thinker) (Xu et al. 2025) Inspired by the powerful capabilities of RL with the verifiable reward (RLVR) (Shao et al. 2024] DeepSeek- Al et al. 2025), our natural initial approach is treat the human preference yo for the pair (41, a2) as a verifiable reward, and launch a RLVR training based on Qwen2.5-Omm -0nni. However, in practice, find that the cntructiio-followiin
reasoning capabilities of Qwen2.5- 5-mni are very weak (more detailed discussions can be found in Appendix E]. Therefore, we adopt a two-stage post-training process ("SFT RL") to develop SpeechJudge- GRM (Figure We describe the details as follows.
SFT Stage We consider SFT as a "cold start" stage to improve the Qwen2.5-Omni's instruction- following, reasoning, and speech naturalness understanding capabilities. We select Gemini-2.5- Flash (Comanici et al. 2025) one of the best-performing closed-soures models on SpeechJudge-
Eval (Table 2 to serve as teacher model, and instruct to generate the CoT data. Specifically, for each sample d = (t, 01, 02, Br) from SpeechJudge-Data we use the CoT prompt from Table (denoted as ICTT) to instruct Gemini-2.5-Flash to generate a rationale-based output (denoted as O )e We then extract the preference judgment (μæ) from this output. For samples where Gemini-2.5-Flash'ss preference is consistent with the human (i.e., VM = V/)), we concatenate the CoT prompt and the model's output, [Ioo,, Dteacher], to create a data point for our SFT dataset. Conversely, we consider the sample d challenging case and reserve the prompt Ico for the second- stage RL dataset. During the SFT stage, for each training sample [Cor, Oteacker), perform the next token prediction only on the segment Oteaheer-
RL Stage We treat the annotated human preference as verifiable reward, and, building on the SET model, we further trained it using the GRPO algorithm (Shao et al., 2024). Specifically, for each sample d (l, 01, 02, 3/H) in the RL dataset, we adopt the COT prompt to instruct the policy model to conduct multiple rollouts during each iteration. For the i-th rollout, we parse the model's preference for (a1,a2). denoted as V'M- Following (Liu et al. 2025c). we use an accuracy- -base rule to calculate the reward: the reward is if y'M VH. and 1 otherwise. In other words, during the RL stage, we only constrain the model's 's final naturalness judgment to align with human preferences, allowing the model autonomously optimize its reasoning and rationale generation capabilities.
We denote the training dataset of SpeechJudge-GRe as SpeechJudge-Date (train). Its construction process is as follows (see Appendix B.2 for more details). Based on the raw SpeechJudge- -Daaa we first filter out all samples at the Full Disagreement (FD) level. For the other samples-as at the FA, WA, and WD levels-we apply a majority voting principle among annotators to determine the final label for each. We then further exclude samples with a "Tie" label, using only the remaining preference data to form the SpeechJudge-Daee during both the SFT and RL stages. Other experimental setup details are provided in Appendix F
(train). We use LoRA (Hu al., (022) to fine-tune the GRM
5.2 EFFECTIVENESS OF SPEECIJUUDEE-GRM NATURALNESS JUDGEMENT
To verify the effectiveness of SpeechJudge-GRM for naturalness judgment, we evaluate it on the SpeechJudge-EE benchmark. We develop SpeechJudge-BBTT -TRM as a baseline, which utilizes the BTRM paradigm (Bradley& & Terry] 1952 Rafailov et al. 2023) by adding linear layer Qwen2.5- Omni-7B 7 (Thinker) to produce a single scalar reward prediction. SpeechJudge-BIT -TRM also uses LoRA fine-tuning and uses the same training data SpeechJudge-CRM.
8

---

Published as a conference paper at ICLR 2026
Model
T-ACC
N-CMOS
Twen2.5-0.5B-TTT
84.0%
0.00
INTP
32.90%
42.90%
24.30%
Speechjudge-Dats
40.80%
33.80%
w/ INTP
87.0%
0.18
w/ Speechjudge (offline)
30.40%
40. 10.60%
SpeechJudge-
91. 91.0%
0.16 sous
w/ SpeechJudge GRM (offline)
91.0%
0.21
w/ Speechhuudee (online)
25.00%
41.70%
33.30%
w/ SpeechJudge- GRM (online)
90.0%
0.25
20%
40%
60%
80%
100%
(a) Text Accuracy (T-ACC) and Naturalness CMOS (N-CMOS).
(b) Win/Lose/Tie of speaker similarity after post-training.
Figure 6: Post-training of Qwen2.5-0.5B-TTS based on SpeechJudge. We display the objective
results (WER and SIM) in Table 14 of Appendix H.2
From the results of Table we can observe that: (1) The SpeechJudge-BTRN achieves a 72.7% %% level of performance comparable the initial development of BTRMs in the textual LLM RLHF field (Stiennon et al.) 2020) Bai et al.
agreement with human preferences on SpeechJudge-Eve
2022 Ouyang et al., 2022). (2) After conducting SFT training with the CoT data, the accuracy of SpeechJudge-Giee MRM (SFT) reaches 75.3%. Besides, further RLVR training improves the final model SpeechJudge-GRMN (SFT+RL) to an accuracy of 77.2% (3) Due to the generative nature of the GRM, we can further enhance the accuracy of SpeechJudge
using inference-time scaling. For example, by using majority voting across 10 outputs instead of just one, the accuracy is improved by approximately 2 percentage points (75.3% - 77.6%; 77.2% + 79.4%). These results collectively verify the effectiveness of our proposed SpeechJudge-GRM for judging speech naturalness.
5.3 HIGL-QUALITY SAMPLE SELECTION BASED ON SPEECHJUDGE-GRM
We investigate the effect of SpeecchJudge-based reward models for high-quality sample selection. We use the hard cases from SeedTTS-Eval (Anastassiou et al., 2024) and the code-switching cases from Amphion-TTS-EE
(Zhang et al.] (2024) as target texts. For each text, we instruct the Qwen2.5- Omni-7B (Talker) (Yang et al.J (024) to generate 100 speeches. We then ask human subjects to compare the best-of-100 output-
selected by either SpeechJudge-BTRA or SpeechJudge- GRM aaa-nsannst-. a randomly sampled output. The evaluation measures the win/lose/tie ratios based on speech naturalness. From Figure 5 we observe that the best-of-100 samples selected by both SpeechJudge-BTTe -TTMM and SpeechJudge-GR.. sample from the same set. This finding demonstrates the advantage of using the SpeechJudge-Dee corpus for training human-aligned reward model. Furthermore, SpeechJudge-GRN exhibits better performance than SpeechJudge- -TRM,, which highlights the superiority of the proposed GRM.
are more likely to outperform a randomly selected
5.4 POST-TRAINING OF ZERO-SHOT TTS BASED ON SPEECHJUDGE-GRY
We investigate the effect of using SpeechJudge-Gre GRMM as a reward function for post-training of TTS model. Specifically, develop a new zero-shot TTS model, Qwen2.5-0.5B-TTS, to serve as the base model, which was not involved in the construction of the SpeechJudge- based on Qwen2.5-0.5B Yang al. (024), adopts the classic two-stage "AR+Diffusion" architec- ture (Anastassiou et al. 2024 Du et al. 2024a). uses the speech tokenizer from DualCodec (Li et al.) 2025a), and pre-trained on the Emilia dataset (He et al. 2025
This model is
Based on this pre-trained model, we design four comparative methods: (1) w/ INTP: We use the intelligibility preference dataset, INTP (Zhang al., (0255), perform offline DPO align- ment (Rafailov et al., 2023). (2) w/ JreeechJJdge-DDee Data: We use the SpeechJudge- -ata (train) to perform offline DPO alignment. (3) w/ SpeechJudge- (offline): We use SpeechJudge-GRe as an offline preference data annotator. We take all speech pairs from the INTP dataset and re- annotate their preference labels using SpeechJudge-GRM. then perform offline DPO alignment on reward function for the online DPO algorithm (Guo et al. 2024). The training data consists of only the prompts from INTP (i.e., the target texts and speech references for zero-shot TTS).
the resulting data. (4) w/ SpeechJudde-GRM (online): We use SpeechJudge-GRe
We use SeedTTS- Evaal (Anastassiou et al., 2024) and Amphion-TTS. -val (Zhang et al. 2025b 2024 (025c) evaluation sets. We present the objective results (WER and SIM) in Table 14 and the subjective results in F We observe that both intelligibility and naturalness are enhanced for all the four methods after post-training. Additionally, the post-training method based on SpeechJudge-

---

Published as a conference paper at ICLR 2026
GRM achieves greater improvement in naturalness (Figure 6a.. Besides, the SpeechJudge-basee methods could match or lead to a slight improvement in speaker similarity (Figure
LIMITATIONS AND FUTURE WORK
While SpeechJudge-De Data and Speechludge-Gre GRM represent a step toward human-alin -ainned speech naturalness judges, several limitations remain and open up directions for future work.
Scope of data and annotators. Our corpus is constructed entirely from synthetic TTS outputs in Chinese, English, and Chinese-Fnglisl code-switching, and our annotators are professional raters in China (native Mandarin speakers with high but still 1.2 English proficiency). As shown in Ap- pendix C.2 inter-annotator agreement is noticeably higher Chinese than English and mixed subsets, indicating that the current dataset primarily reflects the preferences of Chinese and Chinese- English bilinguall listeners, and is tailored to TTS-style read speech rather than spontaneous conver- sation. Extending SpeechJudge-Deee Data to more languages, speaking styles, and listener populations (including native speakers of other languages and more diverse cultural backgrounds) is important direction for building more universal naturalness judges.
Residual failure cases. Our error analysis in Appendix maining mistakes on SpeechJudge-Evee concentrate on some specific trade-offs, such as clean but robotic vs. slightly noisy but lively speech, prosody vs. articulation, and extreme expressive styles like very high-FO emotional speech or whispers. In these regimes the model can over over-weight clean- ness, weight style-appropriate prosody, or become effectively indifferent when preference gaps are extremely small. Future work could incorporate explicit modeling of recording conditions (e.g., background noise), style-aware priors, and targeted augmentation of expressive and cross- lingual examples to better capture these nuanced preferences.
G.3 shows that JueechJudge-GRM'sss
CoT quality and teacher bias. The CoT capability of SpeechJudge- is bootstrapped from a proprietary teacher (Gemini-2.5-Flashh via an SFT stage. Although our analyses in Appendix G.2 suggest that GRM's CoT is largely self-consistent and moderately faithful, the reasoning style still inherits biases from the teacher, and we do not perform large-scale human verification of the interme- diate explanations. An interesting direction is to involve humans more directly in assessing and cu- rating CoT aationales-sinee in spirit to the concurrent SQ-LLM work (Wang al.) (025a)). which incorporates human involvement in CoT annotation example, by collecting naman-written or human-edited analyses, learning from explicit feedback on explanation quality, and explore alternative, fully open-source teachers for bootstrapping.
From coarse-grained to fine-grained naturalness. In this work, naturalness is annotated and modeled at the utterance level: for each sentence, annotators choose which of two speeches is more natural, and SpeechJudge-GRR, outputs single decision per pair. However, in real-world speech, naturalness is often highly non-uniform within an utterance some segments sound very natural while others contain local artifacts, disfluencies, or prosodic issues. Our current formulation does not explicitly localize such fine-grained phenomena. A promising future direction is to collect segment-level or time-aligned human feedback and to train reward models that can produce not only utterance-level judgments but also fine-grat finengrained scores or rationales over time.
7 CONCLUSION
In this work, we tackle the challenge of aligning speech synthesis with human perception of nat- uralness by introducing SpeechJudge: suite consisting of large-scale human preference dataset "SpeechJudge-Dateee a challenging benchmark (SpeechJudge-Eva)) and a generative reward model (SpeechJudge-GHe Our benchmark shows that even strong AudioLLMSs struggle at naturalness judgment, reaching under 70% agreement with human preferences. In contrast, the proposed SpeechJudge-GRM achieves 77.2% accuracy on Speech.uudee-Eval (up to 79.4% with inference- time scaling @ 10), outperforming classic Bradley-Terry reward model (72.7%). We further demonstrate that SpeechJudge-Gree GRM serves as an effective reward function for post-training TTS models, leading to improved perceived naturalness in downstream evaluations. By releasing data, benchmark, and models, we hope to enable further research on human- -ligned speech genera- tion and more reliable evaluation of speech naturalness.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This research adheres to the ICLR Code of Ethics. Our dataset was constructed with feedback from paid professional annotators under fair labor conditions, and the data itself consists of synthesized speech from properly licensed corpora, safeguarding the privacy of all individuals. All human anno- tations were collected from professional annotators recruited by a third-party data annotation com- pany under written informed consent, on synthetic TTS audio only, with collection of personally identifying information. We acknowledge that our models may reflect linguistic biases present in the English and Chinese source data and recognize that generative speech technology has dual-use potential. We do not condone any malicious use of our work, such as the creation of misleading deepfakes.
REPRODUCIBILITY STATEMENT
To ensure full reproducibility, we will publicly release all key resources from this study. This in- cludes the SpeechJudge-Data corpus, the SpeechJudge-Eval benchmark, the trained model check- points for SpeechJudge-GiRe
and the source code for hoth reward model training and downstream experiments. The main paper and its appendices provide detailed descriptions of our methodology, data construction protocols, and experimental setups to allow for the complete verification of our findings. Audio samples are available on our project website.
ACKNOWLEDGMENT
This work is partially supported by the National Natural Science Foundation of China under Grant 62376237, 2023 Shenzhen stability Science Program, Internal Project Fund from Shenzhen Re- search Institute of Big Data (Grant No. T00120230002), and the Program for Guangdong Introduc- ing Innovative and Enterpreneurial Teams (Grant No. 20237T10X044), We thank Yushun Zhang and the anonymous reviewers for their insightful comments and suggestions. We appreciate the efforts of all the subjects during the subjective evaluation.
REFERENCES
Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Jun-Kun Chen, Weizhu Chen, Yen-Chun Chen, Yi-ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao, Amit Garg, Abhishek Goswami, Junheng Hao, Amr Hendy, Yuxuan Hu, Xin Jin, Mahmoud Khademi, Dongwoo Kim, Young Jin Kim, Gina Lee, Jinyu Li, Yunsheng Li, Chen Liang, Xihui Lin, Zeqi Lin, Mengchen Liu, Yang Liu, Gilsinia Lopez, Chong Luo, Piyush Madan, Vadim Mazalov, Arindam Mitra, Ali Mousavi, Anh Nguyen, Jing Pan, Daniel Perez-Becker, Jacob Platin, Thomas Portet, Kai Qiu, Bo Ren, Liliang Ren, Sambuddha Roy, Ning Shang, Yelong Shen, Saksham Singhal, Subhojit Som, Xia Song, Tetyana Sych, Praneetha Vaddamanu, Shuohang Wang, Yiming Wang, Zhenghao Wang, Haibin Wu, Haoran Xu, Weijian Xu, Yifan Yang, Ziyi Yang, Donghan Yu, Ishmam Zabir, Jianwen Zhang, Lyna Zhang, Yunan Zhang, and Xiren Zhou. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras arXiv preprint, abs/2503.01743. 2025.
Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong. Lelai Deng, Chuang Ding, Lu Gao. Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li. Hui Li, Jiaxin Li, Xiaoyang Li. Xingx- ing Li, Lin Liu, Shouda Liu, Sichao Liu, Xudong Liu, Yuchen Liu, Zhengxi Liu, Lu Lu, Junjie Pan, Xin Wang, Yuping Wang. Yuxuan Wang. Zhen Wei, Jian Wu, Chao Yao, Yifeng Yang, Yuan- hao Yi, Junteng Zhang, Qidi Zhang, Shuo Zhang, Wenjie Zhang, Yang Zhang, Zilin Zhao, Dejian Zhong, and Xiaobin Zhuang. Seed-tts: A family of high-quality versatile speech generation mod- els. arXiv prepriní, abs/2406.024 2024.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislay Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson

---

Published as a conference paper at ICLR 2026
Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Saffield-Dodie Danny Hernan- dez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint, 0tos/2004.05862. 2022.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324 1952.
Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, Chao-Har Huck Yang, and Eng Siong Chng. Audio large language models can be descriptive speech quality evaluators. In ICLR. OpenReview.net, 2025a.
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, al. Wavlm: Large-scale self-supervised pre-training for full stack speech processing, IEEE Journal of Selected Topics in Signal Processing, 16(6):1005- 1518, 2022.
Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang. Jian Zhao, Kai Yu, and Xie Chen. F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching. In ACL (1), pp. 6255-6271. Association for Computational Linguistics, 2025b.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training arXiv preprint arXiv:2501.1716: 2025.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan- Jiang Jiang, Krishna Haridasan, Ahmed Omran, Nikunj Saunshi, Dara Bahri, Gaurav Mishra, Eric Chu, Toby Boyd, Brad Hekman, Aaron Parisi, Chaoyi Zhang, Kornraphop Kawintiranon, Tania Bedrax-Weiss, Oliver Wang. Ya Xu, Ollie Purkiss, Uri Mendlovie, Ilai Deutel, Nam Nguyen, Adam Langley, Flip Korn, Lucia Rossazza, Alexandre Ramé, Sagar Waghmare, Helen Miller, Nathan Byrd, Ashrith Sheshan, Raia Hadsell Sangnie Bhardwaj, Pawel Janus, Tero Rissa, Dan Horgan, Sharon Silver, Ayzaan Wahid, Sergey Brin, Yves Raimond, Klemen Kloboves, Cindy Wang, Nitesh Bharadwaj Gundavarapu, Ilia Shumailov, Bo Wang, Mantas Pajarskas, Joe Hey- ward, Martin Nikoltchev, Maciej Kula, Hao Zhou, Zachary Garrett, Sushant Kafle, Sercan Arik, Ankita Goel, Mingyao Yang, Jiho Park, Koji Kojima, Parsa Mahmoudich, Koray Kavukcuoglu, Grace Chen, Doug Fritz, Anton Bulyenov, Sudeshna Roy, Dimitris Paparas, Hadar Shemtov, Bo- Juen Chen, Robin Strudel, David Reitter, Aurko Roy, Andrey Vlasov, Changwan Ryu, Chas Le- ichner, Haichuan Yang. Zelda Mariet, Denis Vnukov, Tim Sohn, Amy Stuart, Wei Liang, Minmin Chen, Praynaa Rawlani, Christy Koh, JD Co-Reyes, Guangda Lai, Praseem Banzal, Dimitrios Vy- tiniotis, Jieru Mei, and Mu Cai. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities. arXiv preprint, abs/2507.06261, 2025.
Chayan Deb, Adway Mitra, Pratik Kumar, Soumya Jahar, and Biplab Das. ParaSpeechCaps: A Parallel Speech-Caption Dataset for Speech-based Image Retrieval, 2024.
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji. Erhang Li, Fangyun Lin, Fucong Dai. Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao. Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui
Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jiawei Wang, Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song. Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen. R. L. Jin, Ruiqi Ge. Ruisong Zhang, Ruizhe Pan, Runji Wang. Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng
12

---

Published as a conference paper at ICLR 2026
Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shut- ing Pan, T. Wang, Tao Yun, Tian Pei, Tianyu Sun, W. L. Xiao, and Wangding Zeng. Deepseek-v3 technical report. arXiv preprint, abs/2412.19437, 2024.
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang. Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingehuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. Deepseek-r1: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint, abs/2501. 12948, 2025.
Heinrich Dinkel, Gang Li, Jizhong Liu, Jian Luan, Yadong Niu, Xingwei Sun, Tianzi Wang, Qiyang Xiao, Junbo Zhang, and Jiahao Zhou. Midashenglm: Efficient audio understanding with general audio captions. arXiv preprint, abs/2508 03983, 2025.
Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint, 0iss/2407.55405 2024a.
Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang. Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang. Shiliang Zhang, Zhijie Yan, and Jingren Zhou Cosyvoice 2: Scalable streaming speech synthesis with large language models. arXiv preprint, abs/2412.10117,, 2024b.
Zhifu Gao, Shiliang Zhang, Ian McLoughlin, and Zhijie Yan. Paraformer: Fast and accurate parallel transformer for autorrrrrgrresivv end-to-end speech recognition. In INTERSPEECH, pp. 2063- 2067. ISCA, 2022.
Zhifu Gao, Zerui Li, Jiaming Wang, Haoneng Luo, Xian Shi, Mengzhe Chen, Yabin Li, Lingyun Zuo, Zhihao Du, and Shiliang Zhang. Funasr: A fundamental end-to-end speech recognition toolkit. In INTERSPEECH, pp. 1593-1597. ISCA, 2023.
Yuan Ge. Junxiang Zhang, Xiaoqian Liu, Bei Li, Xiangnan Ma, Chenglong Wang. Kaiyang Ye, Yangfan Du, Linfeng Zhang, Yuxin Huang, et al. Sagelm: A multi-aspect and explainable large language model for speech judgement. arXiv preprint arXiv:2508.20911 2025.
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexan- dre Ramé, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, and Mathieu Blondel. Direct language model alignment from online AI feedback. arXiv preprint, abs/2402.04792,, 2024.
Haorui He, Zenggiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, al. Emilia: large-scale, extensive, multilingual, and diverse dataset for speech generation. arXiv preprini, 3501.15901, 2025.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR. OpenRe- view.net, 2022.
Wen-Chin Huang, Szu- Fu, Erica Cooper, Ryandhimas E. Zezario, Tomoki Toda. Hsin-Min Wang, Junichi Yamagishi, and Yu Tsao. The voicemos challenge 2024: Beyond speech quality prediction. In SL.7, pp. 803-810. IEEE, 2024.
13

---

Published as a conference paper at ICLR 2026
Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker- Whitcomb, Alex Beutel, Alex Borzunov, Alex Cameey, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Ka- mali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codis- poti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Back, An- gela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang. Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng. Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll I.. Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beau- mont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czamecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, and Dane Sherburn. Gpt-4o system card. arXiv preprint, abs/2410.21276,, 2024.
Shengpeng Ji, Tianle Liang, Yangzhuo Li, Jialong Zuo, Minghui Fang, Jinzheng He, Yifu Chen, Zhengqing Liu, Ziyue Jiang, Xize Cheng, Siqi Zheng, Jin Xu, Junyang Lin, and Zhou Zhao. Wavreward: Spoken dialogue models with generalist reward evaluators. arXiv preprint, abs/2505.0955 009588, 2025.
Zeqian Ju, Yuancheng Wang. Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Eric Liu, Yichong Leng, Kaitao Song, Siliang Tang, al. Naturalspeech 3: Zero-shot speech synthesis with factor- ized codec and diffusion models. In Forty-first International Conference on Machine Learning, 2024.
Jee-weon Jung, Hee-Soo Heo, Hemlata Tak, Hye-jin Shim, Joon Son Chung, Bong-Jin Lee, Ha-Jin Yu, and Nicholas W. D. Evans. AASIST: audio anti-spoofing using integrated spectro-temporal graph attention networks. In ICASSP, pp. 6367-6371. IEEF, 2022.
Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Ta- tiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geof- frey Cideron, Jean-Bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Róbert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, lain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorster Peter, Danila Sinopalnikov. Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky. Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alexan- der Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlic Chen, Charline Le Lan, Christopher A. Choquette-Choo bud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimezak-Plucinnka, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini. Jean Pouget-Abadic, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju-yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli,, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan. Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk. Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Pöder, Sijal Bhatnagar, Sindhu Raghuram Pa- nyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku
CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisen-
14

---

Published as a conference paper at ICLR 2026
Evei, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanse- viero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle K. Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Seulley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clément Farabet, Elena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry (Dima) Lepikhin, Sebas- tian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma technical report. arXiv preprint, abs/2503.19786. 2025.
Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fréchet audio distance: A reference-free metric for evaluating music enhancement algorithms. In INTERSPEECH, pp. 2350-2354. ISCA, 2019.
KimiTeam, Ding Ding, Zeqian Ju. Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, Jun Chen, Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guokun Lai, Qingcheng Li, Yangyang Liu, Weidong Sun, Jianzhou Wang, Yuzhi Wang. Yuefeng Wu, Yuxin Wu, Dongchao Yang, Hao Yang. Ying Yang. Zhilin Yang, Aoxiong Yin, Ruibin Yuan, Yutong Zhang, and Zaida Zhou. Kimi-audio technical report. arXiv preprint, abs/2504.1844 18425, 2025.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. In NeurIPS, 2023.
Jiqqi Li, Xiaolong Lin, Zhekai Li, Shixi Huang, Yuancheng Wang, Chaoren Wang, Zhenpeng Zhan, and Zhizheng Wu. Dualcodec: A low-frame-rate semantically-enhaneee neural audio codec for speech generation. In INTERSPEECH. ISCA, 2025a.
Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Zhi Zhi-Quan Luo, and Ruoyu Sun. Pre- serving diversity in supervised fine-tuning of large language models. In ICLR. OpenReview.net, 2025b.
Alexander H. Liu, Andy Ehrenberg, Andy Lo, Clément Denoix, Corentin Barreau, Guillaume Lample, Jeln-Malo Delignon, Khyathi Raghavi Chandu, Patrick von Platen, Pavankumar Reddy Muddireddy, Sanchit Gandhi, Soham Ghosh, Srijan Mishra, Thomas Foubert, Abhinav Rastogi, Adam Yang, Albert Q. Jiang, Alexandre Sablayrolles, Amélie Héliou, Amélic Martin, Anmol Agarwal, Antoine Roux, Arthur Darcet, Arthur Mensch, Baptiste Bout, Baptiste Rozière, Bau- douin De Monicault, Chris Bamford, Christian Wallenwein, Christophe Renaudin, Clémence Lanfranchi, Darius Dabert, Devendra Singh Chaplot, Devon Mizelle. Diego de Las Casas, El- liot Chane-Sane, Emilien Fugier, Emma Bou Hanna, Gabrielle Berrada, Gauthier Delerce, Gau- thier Guinet, Georgii Novikov, Guillaume Martin, Himanshu Jaju, Jan Ludziejewski, Jason Rute, Jean-Hadrien Chabran, Jessica Chudnovsky, Joachim Studnia, Joep Barmentlo, Jonas Amar, Jos- selin Somerville Roberts, Julien Denize, Karan Saxena, Karmesh Yadav, Kartik Khandelwal, Kush Jain, Lélio Renard I Lavaud, Léonard Blier, Lingxiao Zhao, Louis Martin, Lucile Saulnier, Luyu Gao, Marie Pellat, Mathilde Guillaumin, Mathis Felardos, Matthieu Dinot, Maxime Darrin, Maximilian Augustin, Mickaël Seznec, Neha Gupta. Nikhil Raghuraman, Olivier Duchenne. Pa- tricia Wang, Patryk Saffer, Paul Jacob, Paul Wambergue, Paula Kurylowicz, Philomène Chagniot, Pierre Stock, Pravesh Agrawal, Rémi Delacourt, Romain Sauvestre, Roman Soletskyi, Sagar Vaze, Sandeep Subramanian, Saurabh Garg, Shashwat Dalal, Siddharth Gandhi, Sumukh Aithal, Szymon Antoniak, Teven Le Scao, Thibault Schueller. Thibaut Lavril, Thomas Robert, Thomas Wang, Timothée Lacroix, Tom Bewley, Valeriia Nemychnikova, and Victor Paltz. Voxtral. arXiv preprimi, abs/2507. 2025a.
Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, Xintao Wang, Xiaohong Liu, Fei Yang, Pengfei Wan, Di Zhang, Kun Gai, Yujiu Yang, and Wanli Ouyang. Improving video generation with human feedback. arXiv preprint, abs/2501.1399 13918, 2025b.
15

---

Published as a conference paper at ICLR 2026
Zijun Liu, Peiyi Wang, Rumxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint, abs/2504.02495, 2025c.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR (Poster). Open- Review.net, 2019.
Potsawee Manakul, Woody Haosheng Gan, Michael J. Ryan, Ali Sartaz Khan, Warit Sirichotedum- rong, Kunat Pipatanakul, William Barr Held, and Diyi Yang. Audiojudge: Understanding what works in large audio model based speech evaluation. arXiv preprint, 0es/2507.11705. 2025.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurlPS, 2022.
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In ICML, volume 202 of Proceedings of Machine Learning Research, pp. 28492-28518. PMLR, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly reward model. In NewrlPS, 2023.
Chandan K. A. Reddy, Vishak Gopal, and Ross Cutler. Dnsmos P.835: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In ICASSP, pp. 886-890, IEEE, 2022.
Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. UTMOS: utokyo-sarulab system for voicemos challenge 2022. In INTERSPEECH, pp. 4521 4525. ISCA, 2022.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint, abs/2402.03300, 2024.
simon3000. Genshin Impact Voice Lines 3imon3300//enstiin voice 2023.
https:/mugginnfacc.co//tttttet
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. Learning to summarize from human feedback. In NeurIPS, 2020.
Xu Tan. Neural Text-to Speeech Synthesis. Springer, 2023.
Zhiyuan Tang, Dong Wang, Yanguang Xu, Jianwei Sun, Xiaoning Lei, Shuaijiang Zhao, Cheng Wen, Xingjun Tan, Chuandong Xie, Shuran Zhou, Rui Yan, Chenjia Lx, Yang Han, Wei Zou, and Xiangang Li. Kespeech: An open source speech dataset of mandarin and its eight subdialects, In NeurlPS, 2021.
Paul Taylor. Text-to-speech synthesis. Cambridge university press, 2009.
Andros Tjandra, Yi-Chiao Wu, Baishan Guo, John Hoffman, Brian Ellis, Apoorv Vyas, Bowen Shi, Sanyuan Chen, Matt Le, Nick Zacharov, Carleigh Wood, Ann Lee, and Wei-Ning Hsu. Meta audiobox aesthetics: Unified automatic quality assessment for speech, music, and sound. arXiv preprint, abs/2502.05139) 2025.
Hui Wang, Jinghua Zhao, Yifan Yang. Shujie Liu, Junyang Chen. Yanzhe Zhang, Shiwan Zhao, Jinyu Li, Jiaming Zhou, Haoqin Sun, Yan Lu, and Yong Qin. Speechllm-as aaalllm----judgess Towards general and interpretable speech quality evaluation. arXiv preprint, abs/2510.1 14664, 2025a.
Li Wang, Junyi Ao. Linyong Gan, Yuancheng Wang. Xueyao Zhang, and Zhizheng Wu. Audio deepfake verification. arXiv preprint 2tti::2509.847777. 2025b.
16

---

Association for Published as a conference paper at ICLR 2026
Siyin Wang, Wenyi Yu, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Lu Lu, Yu Tsao, Junichi Ya- magishi, Yuxuan Wang, and Chao Zhang. Qualispeech: A speech quality assessment dataset with natural language reasoning and descriptions. In ACL (1), pp. Computational Linguistics, 2025c.
Xihuai Wang, Ziyi Zhao, Siyu Ren, Shao Zhang, Song Li, Xiaoyu Li, Ziwen Wang, Lin Qiu, Guan- glu Wan, Xuezhi Cao, Xunliang Cai, and Weinan Zhang. Audio turing test: Benchmarking the human-likeness of large language model-based text-to-speech systems in chinese. arXiv preprint, abs/2505.11200, 2025d.
Yuancheng Wang. Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. Maskget: Zero-shot text-to-speech with masked generative codec transformer. In ICLR. OpenReview.net, 2025e.
LLM-Core-Team Xiaomi. Mimo-audio: Audio language models are few-shot learners, 2025. URL httts:///githue com/X Xi aomi MiMo//n Mo-AAuia
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. In NeurIPS, 2023.
Jiazheng Xu, Yu Huang, Jiale Cheng. Yuanming Yang. Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, Jiayan Teng, Zhuoyi Yang, Wendi Zheng, Xiao Liu, Ming Ding, Xiaohan Zhang, Xiaotao Gu, Shiyu Huang, Minlie Huang, Jie Tang, and Yuxiao Dong. Visionre- ward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint, abs/2412.21052 2024.
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang. Bin Zhang, Xiong Wang. Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report. arXiv preprint, abs/2503.20211 2025.
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng. Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint, abs/2412.15115,, 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. DAPO: an open-source LLM reinforcement learning system at scale. arXiv preprint, abs/2503.11470 2025.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agar- wal. Generative verifiers: Reward modeling next-token prediction. In ICLR. OpenReview.net, 2025a.
Xueyao Zhang. Liumeng Xue, Yicheng Gu, Yuancheng Wang, Jiaqi Li, Haorui He, Chaoren Wang, Ting Song, Xi Chen, Zihao Fang. Haopeng Chen, Junan Zhang, Tze Ying Tang, Lexiao Zou, Mingxuan Wang, Jun Han, Kai Chen, Haizhou Li, and Zhizheng Wu. Amphion: An open-source audio, music and speech generation toolkit. In SL.7. IEFF, 2024.
Xueyao Zhang, Yuancheng Wang, Chaoren Wang, Ziniu Li, Zhuo Chen, and Zhizheng Wu. Ad- vancing zero-shot text-to-speech intelligibility across diverse domains via preference alignment. In ACL. Association for Computational Linguistics, 2025b.
Xueyao Zhang, Junan Zhang, Yuancheng Wang, Chaoren Wang, Yuanzhe Chen, Dongya Jia, Zhuo Chen, and Zhizheng Wu. Vevo2: Bridging controllable speech and singing voice generation via unified prosody learning. arXiv preprint arXiv:2508. 16332, 2025c.
17

---

Published as a conference paper at ICLR 2026
Guanlong Zhao, Siyuan Feng, Xinyuan Li, Shiyin Kang, Helen Zhao, and Shinji Watanabe. L2- ARCTIC: Non-native English Speech Corpus. In 2018 JEEE Spoken Language Technology Workshop (SLT), pp. 312-318, 2018. doi: 11.11099SLT.20188833993
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing. Hao Zhang, Joseph Gonzalez, and lon Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In NeurlPS, 2023.
A THE USE OF LARGE LANGUAGE MODELS
We utilized Large Language Models (LLMs) for two assistive tasks: (1) to correct grammar and polish the language of this manuscript, and (2) generate and normalize the target texts for SpeechJudge-Diee Data during the dataset construction. All core scientific contributions, including ideation, methodology, and analysis, are entirely those of the authors.
B DETAILS OF BRGECHUUGGEDATA
B. DETAILS OF PROMPT CONSTRUCTION
7.6%
ARS vs. A66
ARS vs. CosyVoice2
ARS F3-TTS
2.4%
9.6%
MaskGCT
CosyWlce2
ConWeice22
9.5%
CosyVocee--NTT CosyVaice2-INT
ConyVeica2-INTP Ints-INTP
15.2%
F5-TTS MaskGCT
Ints-INT vs. Ints-INTP
7.8% 6,9%
MuskGCT va. MaskGGT
Figure 7: Distribution of the speech pairs of Speechhuuge-Data. The chart illustrates the percentage of both intra-model (c.g., ARS vs. ARS) and inter-model (c.g., ARS vs. Cosy Voice2) pairs.
For the target texts paired with the regular speech references, we use DeepSeck- V3 (DeepSeck-A) al. 2024 to fix typos and normalize punctuations, the prompt used is listed below.
System Prompt:
ASR models. Please help clean up (e.g.. correct typos, add proper punctuation marks, and make sentences semantically coherent). Note: (1) You can modify, add, or replace words that better fit the context to ensure semantic coherence. (2) Please only return the cleaned-up result without explanation.
I obtained text from audio file based
User Prompt (Example):
panda eats shoes and leaves
System Output (Example): panda cats shoots and leaves.
For the target texts paired with the expressive speech references, we use DeepSeek-V V3 to generate several scripts in different writing styles based the speech reference's text, the prompt used is listed below.
System Prompt:
; 7 * * (AX: Α #A RNRRS AX. Formal, Informal,
# (#2) $ æd.
18

---

Published as a conference paper at ICLR 2026
SpeechJuudge-Daa Data (raw) (O8K)
SperchJudge Eval
Phoneenne Seeccin
(1K)
Sinil Bidd sampling from FA-lanst sumplia
Speech-udge-Dat (pref) (79K)
Speechludige Data (hq) (440)
samples
Speech
(1K)
Plemai data
Specchuudde (trai)
(42K)
Figure From the raw Speech/udge- Data to its different subsets.
Literary, Journalistic, Advertising, etc.)
* #
19/ TH 22 FSON
"contimue":
("type": M "content": "M
("type": " "content": "
10
].
"random": "
"translation": "# 2F J. J4 ,
"codeSwitching". " "franddom/i Ff TH FEcde- swicce-switching. M X
User Prompt (Example):
panda eats shoots and leaves.
System Output (Example):
"contimue":
("type": "Formal", "content": "This behavior is characteristic of the species Ailuropoda melanoleuca, which sustains itself primarily on herbivorous diet consisting various bamboo species. The consump- tion of both young shoots and mature leaves is essential for meeting its nutritional requirements.") ("type": "Informal", "content": "And then, after chowing down on all that bamboo, it just kinda chills maybe takes long nap something. They't re pretty laid-back animals, you know? Not much else going on when you'te full leaves."").
("type": "Fantasy", "content": "Legend says that with every shoot consumed, a little more of the forest's s ancient magic is absorbed into its being. The leaves it leaves behind are not mere waste, but enchanted tokens that heal curse, depending on the heart of the finder.").
J.
"random": "Legend says that with every shoot consumed. a little more of the forest's ancient magic is absorbed into its being. The leaves it leaves behind not mere waste, but enchanted tokens that can heal curse, depending the heart of the finder."
"translation". "f 32
ERE F #F
# $0. IT
"codeSwitching". "Legend says that with every shoot consumed, little more of the forest's ancient magic uXinto its being. The leaves leaves behind not waste, but enchanted tokens that heal elccurse depending on the finder #0ear..""
19

---

Published as a conference paper at ICLR 2026
B.2 SUBSETS OF SPEECHJUDGE-DEE DATA
We begin with the SpeechJudge-Data (raw) corpus, containing 99K pairs, where each pair is annotated by multi- ple labelers as a five-scale naturalness CMOS. We aggregate these annotations via a majority vote for each pair, and subsequently discard all "Tic" pairs, yielding the 79K-pair human preference data, denoted as SpeechJudge-Datte (pref).
We construct several subsets based on SpeechJudge-Data (Figure 8
During our preliminary analysis based on SpeechJudge-Datt (pref), observe that a significant disparity in intelligibility between two speech samples can overshadow the subtler quality of natu- ralness, biasing human preference toward the more comprehensible sample. To mitigate this con- founding factor and create more high-quality dataset focused specifically on naturalness, we fur- ther refine the data. Specifically, we retain only pairs where the absolute WER gap of those is below 12%. This process results in the 4pir high-quality SereechJudge-Dare (hq) subset, ensuring that its preference labels are more reflective of genuine differences in naturalness.
From SpeechJudge-De Data (hq), we construct our benchmark, SpeechJudge-Evv by applying strat- ified sampling to FA-level pairs, resulting in 1,000 pairs; its composition detailed in Table 4 Similarly, we the same strategy construct validation set of the same size, SpeechJudge- Data (dev). The remaining 42K pairs, SpeechJJugee-Die (train), constitute the training set for our reward models.
Table 4: Distribution of the SpeechJudge-Fvv Eval benchmark.
Subset
Source Speech References
Languages Target Texts
Pairs
200
Regular
Emilia-Large
zh
200
PuraSpeechCaps, L2-Aretic. KeSpeech, Genshin, etc.
en
200
Expressive
zh
200
mixed
200
C HUMAN ANNOTATION DETAILS
The complete annotation guidelines are attached below:
TASK INTRODCUTION
each task, you are required complete two evaluations:
1. Pronunciation Error Detection
For each audio clip, we provide the target text that is intended to be read aloud. You need to determine if there any pronunciation errors in the audio, such omissions (missing words), insertions (extra words), or substitutions (wrong words).
2. Naturalness Comparison Compare audio clips and determine which sounds natural and like real human speaking.
Attention:
Please headphones for the evaluation better capture audio details and improve judgment accuracy.
ANNOTATION CRITERIA
A. PRONUNCIATION ERROR DETECTION
Pronunciation emrrr include the following three categories:
Omission: Certain words from the target text are missed.
Insertion: Extra words not the target text are added, c.g., repeating words.
Substitution: Certain words are misread, e.g. reading names, numbers, polyphonic characters, other words incorrectly, making word order emrors.
Attention: These pronunciation emrors can occur at any point in the audio. For example:
20

---

Published as a conference paper at ICLR 2026
At the beginning of the audio, words are spoken that are not in the target text.
end of the audio, some content from the target text is omitted.
In the middle of the audio, omissions, insertions, substitutions occur.
В. NATURALNESS COMPARISON
Natural speech should sound like real person talking. Specific criteria include:
The audio clear, free from rohoticfelectronic shouting, or other irrelevant background voices).
obvious noise (e.g., unnatural laughter,
The intonation is natural and expressive, not flat mechanical.
The speaking rhythm is reasonable, with moderate speed and appropriate pauses. (Note: appropriate pauses affect the naturalness of the audio but are not classified as "pronunciation errors").
Word stress is placed correctly, conforms normal linguistic sense, does not sound abrupt.
The rating scale is follows:
A +2: Audio significantly more natural than (large difference).
A +1: Audio A slightly more natural than B (slight difference).
Tie: The naturalness of the two audio clips similar and difficult to judge.
. B +1: Audio B is slightly more natural than A (slight difference).
B +2: Audio significantly more natural than (large difference).
Attention:
If are minor errors individual words, please do not let them affect your overall judgment
of naturalness.
However, if large number of content found that severely interfere with your listening experience, this will affect the audio's naturalness.
C.I INDIVIDUAL ANNOTATOR RELIABILITY
To assess the reliability of individual annotators, we computed agreement rate for each participant. This rate measures the extent which an annotator's judgments align with those of their peers on the same sample (1, 01, 42).
For given sample annotated by group of M annotators, the agreement score for annotator is calculated as the fraction of the other M - annotators who assigned the exact same label. An annotator's final reliability score is the average of these scores across all samples they evaluated. We excluded participants who annotated fewer than 10 samples from this analysis.
Formally, for an annotator who labeled N; samples, the agreement rate Tiy for sample j is defined as:
Σ Uyns = yyl
Tig
M
The overall agreement rate for annotator i, denoted as Rú is then:
R,
Σ
T
where Mig E B,T) is the label assigned by annotator to sample j. The label T (i.e., Tie) is treated distinct category, and it's agreement is counted only on exact matches.
Figure illustrates the distribution of these agreement rates for our 69 annotators for SpeechJudge- Data (raw). The distribution is generally unimodal with peak the 60-70% range") which indi- cates a consistent and reliable level of performance across the annotation pool.
We have noted that one annotator's agreement with the others is less than 30% ultimately removed his data from SpeechJudge-Datt.
21

---

Published as a conference paper at ICLR 2026
Regular
Expressive
24--
24
24-
Number of AAmneeattr
21
23
21
20
18
18
15
25.
15
12-
12
12
18
10%
30
30%
80%4
130%1
10% 30%1
(30%, 30%
30%A
(00% 7086
(10%, 90%4
(wos, 13044
10% 1098
(10%. 20%0
(3096 40%0
(50% 60%A
E10%, 80WA
100%% 100%
10%,
(00%
(40%,
100
(10%
(50%
1909.
(30%
(30%
(40%
150%
(00%
(10%
Figure 9: Distribution individual annotator agreement rates. Most annotators fall between 50%
and 80%, typically peaking in the 60-70% range.
C.2 INTER-ANNOTATOR AGREEMENT
Complementary to the per-annotator reliability analysis in Appendix C.I we now quantify the over- all level of agreement among annotators at the dataset level. Following common practice in RLHF- style preference datasets, we compute inter-annotator agreement as the probability that two anno- tators chosen at random assign the same preference label to the same pair (Stiennon et al. 2020 Ouyang et al. 2022 Xu et al. 2023). summarzis the results.
Ternary vs. binary preferences. Speechludge- (raw) contains ternary labels (i.e., A bet- ter / B better / Tie), and the corresponding inter-annotator agreement is 50.7% + 0.1% over the whole corpus. After removing "Tie" cases and restricting to clear binary preferences (SpeechJudge-Date (pref), labels A better / B better only), the agreement rises to 69.0% + 0.2%. This level is comparable to well-established RLHF datasets text and vision: ImageReward re- ports agreement of 65.3% = 1.6% (Xu et al. (203)), and InstructGPT reports 72.6% + 1.5% on binary A/B comparisons (Ouyang et al. 2022). These numbers indicate that, despite the inherent subjectivity of the task, our binary preference data are on par with existing human-feedback corpora.
Effect of style. Within each language, expressive prompts are slightly harder than regular ones: for example, SpeechJudge-Data (pref), regular zh reaches 0.2%% agreement while expressive zh reaches 73.0% + 0.6%. A similar, mildly lower agreement is observed for expressive en and mixed subsets. Overall, however, regular vs. expressive speech remains in a similar agreement range, suggesting that our guidelines allow annotators to make consistent naturalness, judgments.
Effect of language and code-switching. In both the raw ternary data and the binary preference subset, agreement on Chinese (zh) is notice- ably higher than on English (en) and especially on code-switched (mixed) samples (e-g.. 74.4% vs. 61.7% and 62.5% in SpeechJudge- Judge-Data (pref)). We attribute this gap to the annotator population: our annotators are predominantly native Mandarin speakers; while English and code-switched items are assigned only annotators who pass high English proficiency bar, they are still L2 or bilingual listeners. Similar phenomena-
A more pronounced pattern emerges across languages.
agreement for L2 or cross-lingual annotations compared to ones have also been discussed in recent text RLHF studies (Bai et al. 2022).
In the current work, we therefore interpret SpeechJudge- Data as primarily reflecting the preferences of Chinese and Chinese-Enge English bilingual listeners. In future work, we plan to augment the corpus by recruiting native English speakers for the English subset and stronger bilingual/native speakers for code-switched data, as to improve agreement these subsets and broaden the cultural and linguistic coverage of the dataset.
C.3 INTELLIGIBILITY ANNOTATION ANALYSIS
We provide detailed analysis of the relationship between the mostly common used objective intel- ligibility metric, Word Error Rate (WER), and the subjective human judgments of intelligibilityy Our goal is to determine the extent which WER can serve a reliable proxy for human perception.
We use all the speech samples from SpeechJudge-Deee (raw) for this analysis. We visualize the relationship between WER and the subjective text accuracy in Figure T0 For the regular speeches
22

---

Published as a conference paper at ICLR 2026
Table 5: Inter-annotator agreement (mean + std, in %) on SpeechJudge-Deee and prior human preference datasets in the filed of RLHF.
Regular
Expressive
Dataset
Preference
Total
en
mixed
SpeachJudge-Dase (raw)
Temary
54.9 aaw
55.5 gase
50.2
49.5 2im
36.3 sam
50.7
2e1s
SpeechJudge- (pref)
61.7 g3w
74.4
gat
62.5
69.0
ImageReward (Xu
2023
Binary
65.3 =5
InstructGPT [Ouyang et 2022
72.6
2
SpeechJudge Data (raw) lernary labels (A beller beller / Tiel: SpeechJudge Data (pref) lies and binary labels (A better / better). Results SpeechJudge-Doee are further broken down by style and language.
(the orange curve), we observe a consistent negative correlation: the WER increases, its perceived text accuracy steadily declines, For the expressive speeches (the green curve), the similar trend holds for expressive speech when WER is under about 12%. When WER is over the threshold, however, the correlation between WER and the subjective text accuracy weakens significantly. We think this divergence is sourced from that the greater stylistic variations in expressive speech pose substantial challenge to the robustness of ASR systems compared to the regular samples.
begins to on expressive samples.
Average Text Accuracy
80%
70%
60%
50%
All
Regular
15
2%
95 10% 118 12%
13% 14%
15% 10% 17% 18% 19% 20% 21% 22% 23% 24% 25% 26% 27% 20% 29% 30%
Figure 10: The relationship between the ioman-annotated text accuracy and WER.
D DETAILS OF EVALUATION ON THE SPEECHJUDGE-EVAN BENCHMARK
During the evaluation on the SpeechJudge- Benchmark of Table protocol for each model:
we adopt the following
WER (Radford et al. 2023 Gao et al. 2023 We employ Whisper large-rreeee
Radford
et al. 2023) for English texts, and Para formorrr-rrooooo Gao et al. 2022 2023 for Chinese and code-switching texts.
SIM
Chen et al. 2022): : We compute the cosine similarity between the WavLM TDNN 6 Chen 2022) speaker embeddings of generated samples and the prompt samples.
al.
FAD (Kilgour al. 2019); We use the officially released checkpoint, VGGish, to obtain the
FADs audios.
DNSMOS (Reddy et al. 2022 We use the officially released scrip to calculate the DNSMOS of audios.
CE, CU, PC, and PQ Tjandra et al. 2025 : We use the officially released toolki to get the
predicted MOS for audios.
htter//ttttttimmoooooiolllllaaat local.py
23

---

Published as a conference paper at ICLR 2026
AASIST (Jung al., ) It is common baseline model for audio deepfake detection, em- ploys heterogeneity aware approach to integrate spectral and temporal sub-graphs. We use a large-scale in-house corpus to train the model.
ADV (Wagg et al. 2025b:: is a state-of-the-art (SOTA) deepfake detection model built upon the pre-trained w2 w2v-bert-2.. berbbb.rtt20 It utilizes multi-task training approach involving deepfake source tracing to extract robust audio deepfake features. We use the same corpus of AASIST to train the model.
We use the plain prompt of Table to instruct AudioLLMs to pairwise score the naturalness of two audios. For the closed-souree models, we use the official API released by Google 10 for Gemini and OpenA T for GPT. We use the model variants gemi ni 11-22.2 5-fffashh gemini- pro.
AMs;
and gpt 1:-4-oiiiioooo audio-previeiee wwwppeeeewee0000 06 for Gemini-2.5-Flash, Gemini-2.5-Pro, GPT-4o mini Audio, and GPT-4o Audio.
spt -0 Mooommnn--- -uuii- preview- 2024-12 12-17,
E MORE EVALUATION RESULTS OF EXISTING AUDIOLLMS
Using AudioLLM as a judge models, prompt engineering strategies are usually believed crucial for improving the performance (Zheng et engineering strategies include using the CoT prompts to activate the model's thinking and reason- ing abilities (Zhang et al.) 2025a Liu et al. 2025c, Manakul et al.) 2025), or employing few-shot evaluation formats (Zheng et al. 2023
2023 Manakul et al., 2025). Some common prompt
Xiaomi 2025).
E.1 CHAIN-OF-THOUGHT PROMPTING FOR AUDIOLLM JUDGES
In this study, we investigate whether using the CoT from Table T helps AudioLLMs better judge speech naturalness. We display the results in Table 6 Interestingly, find that some closed- source AudioLLMs, such as Gemini-2.5-Flash, improve their performance on SpeechJudge-Eve through this thinking and reasoning process. However, this strategy often does not work for existing open-source AudioLLMs.. For example, the results Table [ show that Qwen2.5-0mmni778 and Kimi-Auddoo7BBInntruu which is already the leading open-source models on SpeechJudge-Eval (Table 2). actually sees decline in performance when using the CoT prompt.
Table Performance AudioLLMs on SpeechJudge--E Evall when using the CoT prompt of Table
Model
Regular
Expressive
Total
Qwen2.5-0mmi-7B
62.0
59.7
60.6
w/ CoT prompt
54,4
50.6
52.9
Kimi-Audio-7B-hmee 7B--sttutt
65.5
68.0
67.0
w/ CoT prompt
67.4
66.1
66.5
Gemini-2.5-Fla. CoT prompt
73.5
66.2
69.1
75.0
67.5
70.5
Based on our preliminary qualitative analysis, we believe the reason why the open-source Audi- oLLMs do work well with the CoT prompt is that their foundational capabilities relatively weak. These weaknesses include cestruction-followiii understanding, long-conter processing, and reasoning abilities. This also why, when we devel- oped SerchhJude-GR.... we did not directly apply RLVR on top of Qwen2.5- 5-2.5--0mi---- 7B.. Instead, we used an initial SFT stage as a cold start.
(such as format-following) multiple-audio
24

---

Published as a conference paper at ICLR 2026
E.2 NEW-SHOT PROMPTING FOR AUDIOLLM JUDGES
Motivated by the common belief that in-context examples can improve LLM judging ability (Zheng
al. 2023; Manakul al. 2025), we also investigate few-shot prompting for Qwen2.5-0mnis
on SpeechJudge-Eval.
Table 7: Performance of Qwen2.550mmii77B on SpeechJudge-Fvve Eval with different k-shot prompts.
Model
Regular
Expressive
Total
Qwen2.5-0mmii77 (0-shot)
62.0
59.7
60.6
w/ 2-shot
50.9
46.6
48.2
w/ 4-shot
46.1
52.0
49.6
w/ 6-shot
50.3
53.0
51.9
w/ 8-shot
51.0
54.8
53.3
w/ 16-shot
48.6
53.0
51.3
We start from the plain zero-shot prompt in Table T which asks the model decide which of two audios is more natural. For the k-shot setting, we prepend k preference exemplars to this prompt. Each exemplar consists of: (i) a target text, (ii) an audio pair associated with this text, and (iii) the corresponding human naturalness label (which of the two audios is preferred). The model is then queried on a new SpeechJudge-Fe Evaa pair with the same instruction and output format, We evaluate Owen2.5-0mni-4 with & {2, 4, 6, 8, 16); results are reported in Table 7
Contrary to the usual expectation that few-shot prompting should help, we observe that none of the k- shot configurations improves over the zero-shot baseline. On the contrary, the overall accuracy drops from 6% in the O-shot setting to 48 53% with few-shot prompts. The degradation is particularly pronounced on regular speech, while expressive speech shows small, inconsistent fluctuations.
These findings are consistent with our observations in Appendix E.1 on chain-of- of-thu prompt- ing: current open-source AudioLLMs such as Qwen2.5-0mni- still have limited instruction- following, multi-audio understanding, and long-context handling capabilities. Although few-shot prompts provide more information in principle, the model struggles to reliably associate multiple text-audio pairs with their labels in the context. As result, increasing k mainly adds complexity the input without yielding better judgments. This further motivates our choice move beyond pure prompt engineering, and instead train dedicated reward models (BTRM / GRM) on human preference data for robust naturalness evaluation.
F TRAINING DETAILS OF SPEECHJUDGE-GRM
SFT Stage We use Gemini-2.5-Flash (Comanici et al. 2025) to generate the CoT data for SpeechJudge-Deee (train). For the total 42K samples, Gemini-2.5 2.5--22ss- judgments agree with human feedback on 25K samples, while they disagree on 17K samples. During the SFT stage, we fine-tune Owen2.5-0mmi-775 (Thinker) (Xu et (225) on the 25K CoT data using LoRA (Hu al.) 2022) with rank of 128. We use Adam (Kingma & Ba 2015 Loshchilov & Hutter 2019) as the optimizer and set the learning rate to 5e-5. The maximum number of tokens per batch is 4000. We select the best checkpoint SpeechJudge-DDa (dev) as the SFT model, SpeechJudge-GRI (SFT).
use the 17K samples (as described above) to conduct DAPO YYu et al. 2025) ) which is an enhanced variant of GRPO (Shao et al. 2004). We utilize the ms- swi ft toolkit to launch the training process. We initialize the policy model with the SFT model and use LoRA training with a rank of 64. The number of rollouts for each prompt is set to 8, and the batch size is 32. The learning rate is 5e-6. We select the best checkpoint on SpeechJudge- (dev) as the final SpeechJudge-GRM model, i.e., Speechludge-GRM. (SFT+RL).
RL Stage
25

---

Published as a conference paper at ICLR 2026
Table Accuracy of different models on SpeechJudge-Eve (regular) across language settings.
Regular
Model
en2en
zh2en
zh2zh
en2zh
Avg
Owen2.5-Ommi-77
48.1
58.5
75.7
66.0
61.0
Gemini-2.5-Flas
59.4
62.8
81.6
87.6
72.8
SpeechJudge-BTTe
66.0
71.3
86.4
86.6
77.5
OpeechJudge-GRN (SFT)
67.0
74.5
84.5
85.6
77.8
w/ Voting 10
65.1
75.5
83.5
85.6
77.3
SpecchJudge-GRi (SFT+RL)
69.8
77.7
85.4
83.5
79.0
w/ Voting 10
75.5
79.8
80.6
86.6
80.5
Table 9: Aceuracy of different models on SpeechJudge- (expressive) across language settings.
Expressive
Model
en2en
zh2en
zh2zh
en2zh
en2mixed
zh2mixed
Avg
Owen2.5-Omnii-1 Gemini-2.5-Flash
43.6
51.7
61.1
70.9
64.5
69.6
59.7
53.6
68.3
73.3
76.4
64.5
67.1
66.2
OpeechJudge-BBRR.
62.9
56.7
72,2
85.5
68.6
67.1
69.5
Speechludge-GRN (SFT)
61.4
66.7
89.1
77.8
74.4
73.4
73.7
w/ Voting 10
69.3
75.0
88.9
90.9
71.1
74.7
77.8
Speechludge-GRNN RSFT+RL)
71.4
81.1
70.2
81.0
76.0
w/ Voting
75.0
66.7
82.2
89.1
72.7
84.8
78.7
G MORE EVALUATION RESULTS FOR SPEECHJUDGE-GRM
G.1 PERFORMANCE UNDER DIFFERENT DATA DISTRIBUTIONS
In Table of the main paper, we reported accuracies on SpeechJudge-HE -aaee-vall aggregated over all languages and styles. Here we provide a more fine-grained view of how different judges behave under different data distributions, and we additionally evaluate completely -iitrii-ittiitto (OOD) test set involving real human speech versus commercial TTS clones.
Languages on the regular subset. Table 8 breaks down performance on the regular subset of SpeechJudge- by language setting: en2en, zh2en, zh2zh, and en2zh. Across all models we ob- serve that pairs involving Chinese (zh2zh and en2zh) are consistently easier than purely English pairs (en2en). For example, SpeechJudge-GRR. (SFT+RL) reaches 85.4/83.5% accuracy on zh2zhlen2zh, but only 69.8% on en2en, and the same trend holds for BTRM and Gemini- 2.5-Flash We believe several factors contribute to this gap. On the data side, as shown in Appendix C.2 Chinese (zh) subsets exhibit higher inter-annotator agreement than English and mixed subsets, supervision for English-like conditions is more varied and harder to fit. On the modeling side, current TTS sys- tems may also produce relatively high-quality English outputs compared to Chinese, making the naturalness differences between two English samples more subtle and thus more difficult to judge reliably. Even under these challenges, SpeechJudge-GRe (SFT+RL) with Voting @ 10 still achieves the strongest average accuracy (80.5%) among all open judges.
Languages on the expressive subset. Table shows the same breakdown for the expressive sub- set. As expected, expressive speech is generally harder: all models are few points lower than on their regular counterparts. The language pattern persists: Chinese-involving settings (zh2zh, zh2mixed) tend to be easier than en2en. For instance, Speech.luge-GGRM (SFT+RL) with Vot- ing@ 10 attains 82.2% on zh2zh and 84.8% on zh2mixed, compared to 75.0% on en2en. This is consistent with the inter-annotator statistics in Appendix C.2 where expressive and code-switched English subsets show lower human agreement, and also reflects the increased linguistic and prosodic complexity of expressive and code-switched speech. Overall, expressive data slightly more chal- lenging than regular data, but the relative ranking of judges is stable and GRM maintains a clear advantage over BTRM and the AudioL baselines across all language settings.
26

---

Published as a conference paper at ICLR 2026
Table 10: Accuracy of different models on SpeechJudge-Eval across prompt styles.
Model
Regular
Emotional
Accented
Whisper
Game
61.0
56.7
64.4
57.1
61.3
Gemini-2.5-Flass
72.8
63.7
66.7
74.6
66.0
Opecchludge-BTRI
77.5
69.8
71.3
76.2
66.8
Speechludge-Gie (SFT)
77.8
75.3
80.5
71.4
70.2
w/ Voting @ 10
77.3
76.7
80.5
74.6
78.7
Speechludge-GRe GRM RSFT+RL)
79.0
74.4
81.6
79.4
74.5
w/ Voting @ 10
80.5
78.1
85.1
82.5
75.7
Table 11: Accuracy of different models on the out-of-distributio human vs. SeedTTS clone eval- uation set. Each test pair consists of human recording and SeedTTS voice clone for the same sentence; the model must decide which of the two is more natural. In all pairs, the human recording treated the ground-truth natural sample.
Model
Characterl
Character2
Avg
Deepfake Detectors
AASIST ADV
97.2
100
98.6
99.6
100
99.8
AudioLLMMs
Owen2.5-0mmii77 Gemini-2.5-Flash
48.0
44.8
46.4
85.2
85.6
85.4
52.8
48.8
50.8
Naturalness Reward Model
SpecchJudge-BTRN Speechludge-GRM (SFT)
55.6
45.2
50.4
37.6
44.0
40.8
w/ Voting @ 10
36.0
41.4
38.7
SpeechJudge-GRM (SFT+RL)
57.6
67.2
62.4
w/ Voting @ 10
59.8
67.5
63.7
Different prompt styles. Table 10 compares performance across prompt styles: regular, emo- tional, accented, whisper, and game-character speech. For SpeechJudge-BTRM we see that regular prompts are the easiest (77.5%), while emotional and game styles notably harder (69.8% and 66.8%). J. In contrast, SpeechJudge-GRN. benefits more from the expressive settings: with SFT+RL and Voting@ 10, GRM achieves 80.5% on regular, but rises 85.1% on accented and 82.5% on whisper prompts, and narrows the gap emotional and game prompts (from about 10 points for BTRM to only a few points). We believe this reflects the advantage of the generative reward model and its CoT-based training: by explicitly reasoning about artifacts, prosody, and style, GRM is better able to handle challenging conditions such as accented speech and whisper, rather than overfitting to the most frequent regular style.
OOD evaluation on real speech vs. commercial TTS clones. Finally, Table II presents a new, fully diitr--diittibution evaluation designed to stress-test generalization to real speech and unseen TTS architectures. We select two native English voice actors, each recording 250 utterances (500 in total), and commercial SeedTTS voice-cloning A synthesize clones for each utterance. SeedTTS a state-of-the-art proprietary system whose output quality typically higher than that of the open-source TTS models used to construct SpeechJudge- probes the gap between very strong modern TTS and human recordings. For every sentence, we form pair (human recording vs. SeedTTS clone) and treat the human recording as the ground- truth more natural sample. Neither the human recordings nor the SeedTTS outputs are present in SpeechJudge- Data, making this challenging OOD test. The results show three interesting trends:
so this benchmark effectively
First, deepfake detectors (AASIST and ADV) achieve almost perfect accuracy (about 99%) on this task, even though they perform at chance level on SpeechJudge-Eot Eval (Table 2). confirming that they mainly learn to discriminate real vs. synthetic rather than judge naturalness between two synthetic samples.
27

---

Published as a conference paper at ICLR 2026
Second, among AudioLLMs,, Banii-uudo--7-Instruu performs strongly (85.4% average), possibly because its training includes tasks or signals related to authenticity detection. In con- trast, Gemini-2.5-Flash attains only 50.8% accuracy, roughly at chance level and similar to Qwen2.5-0mni-77 (46.4%). This indicates that Gemini's strong performance on SpeechJudge- Eval (69.19 % in 2 does not automatically transfer to the numan-vs-clone setting: it appears to be a good judge for syathetic--s-symhheii naturalness comparisons, but it is not explicitly bi- ased toward humans when facing a high-quality commercial clone.
Third, for naturalness reward models, SpeechJudge-BITKl B is close to random guessing (50.4%), suggesting that the classical Bradley-Terry training may overfit more heavily the specific syn- thetic generators in SpeechJudge-aata Interestingly, SpeechJudge-GRM (SFT) alone performs even worse than BTRM, which we hypothesize is due to SFT encouraging the model over- memorize our prepared CoT patterns and thus hurting OOD generalization (Li eL at. 2025b Chu al. 2025). Once we add the RLVR stage, however, SpeechJudge-Gre GRM (SFT+RL) improves substantially to 62.4% (and 63.7% with Voting @ 10), outperforming BTRM well as generic AudioLLLMs such as Qwen2.5-Omni-77 and Gemini-2.5-Flash on this benchmark. Given that the SeedTTS clones SpeechJudge-Giie has the potential to provide useful feedback not only for open-source TTS models, but also for strong proprictary systems that were never seen during training. This sug- gests that the generative reward modeling paradigm is more robust to distribution shift than clas- sical BTRM and off- fff-the-shelf AudioLLM judges, and that RL human preferences is crucial for recovering and enhancing generalization beyond the synthetic training distribution.
already very close to human quality, this performance indicates that
G.2 QUALITY ANALYSIS CHANN-OF-THOUGHH REASONING
Beyond scalar accuracy, further analyze the quality of the Chain-of-TTe Thhough (CoT) rationales produced by different judges. Specifically, we compare Gemini-2.5-Flash (teacher), SpeechJudge- GRM (SFT), and SpeechJudge-GiRR (SFT+RL) along three aspects: logical consistency between reasoning and conclusion. (ii) faithfulness and hallucination rate as judged by human experts, and (ii) differences in reasoning style.
Consistency between reasoning and conclusion. We first ask whether a model's CoT reasoning logically compatible with its final decision. Using DeepSeek-V3. (DeepSeek-Al meta-judge, we prompt it with the instruction like: given the CoT analysis of A and B (over prosody, pacing, articulation, and overall naturalness) and the final scores assigned the conclusion is consistent with the reasoning. DeepSeek-V. V3 returns binary label (consistent / not consistent) and brief justification. The specific instruction is as follows:
al. 2024
and B, decide whether
System Prompt:
I am having model decide whether A B is better using Chain-of Thainn-of-Thought (CoT)) process. Now, I need you to help determine whether the model's CoT output-its reasoning process and its final conclusion (which is presented as "A: points, B: points") consistent.
Please note:
You only need check whether, in the CoT reasoning, if the model analyzes that A is than B, then in the final scores should also be lower than As long as this condition is met, you can regard it as consistent. Conversely, if the model reasons that A is better than B but assigns A lower score than B in the final output, then it is not consistent.
You only need to return a JSON string with two keys.
The first key "result" with value of 1-0 means consistent, 1 means consistent.
o The second key is "reason", where you briefly explain your reasoning in English.
User Prompt (Example):
Output A:
The intonation is somewhat flat and lacks natural variability, making it sound bit robotie. There are some unnatural stress patterns, and the overall prosody doesn't convey the meaning with human- like nuance.
Prosody and Intonation:
Pacing and Rhythm: The pacing is a bit slow and deliberate, and there's a slight unnatural pause before "They will frequently...". The rhythm feels somewhat monotonous.
28

---

Published as a conference paper at ICLR 2026
Table 12: Consistency between Mhain-of-Thougin (CoT) reasoning and final conclusion for different models on SpeechJudge-Byee Eval, as judged by DeepSeek- V3. A prediction is counted as consistent when the preference implied by the CoT analysis matches the model's final decision.
Model
Consistency
Gemini-2.5-Flassh
97.9%
SpeechJudge-GRMN (SFT)
97.6%
SpeechJudge- (SFT RL)
98.2%
Table 13: Human-rated quality of Chain-of- Chain-of-Thought (CoT) explanations for different models SpeechJudge-Eve Scores (1-3, 1) are assigned by expert raters on three Snnenniioss-Prosoee & Intonation, Pacing & Rhythm, and Articulation & Marity-matching the dimensions specified in our CoT prompt (Table IT
We report the mean and its 95% confidence interval for each dimension.
Prosody & Intonation
Pacing & Rhythm
Articulation & Clarity
Model
Avg
Gemini-2.5-Fiasr SpeechJudge-GRM (SFT)
1.90
2.00
2.10 2n0s
2.00 10
2.00
2.15
1.95
2.03
SpeechJudge-GRM (SFT+RL)
2.10 =0.30
2.00 2021
2.40
2.17
#038
While most words are clear, there is a very noticeable and significant mispronunciation of "will" as "woof" ("they woof frequently make a show"). This is a major error that severely impacts naturalness.
Articulation and Clarity:
Overall Naturalness: The "woof" mispronunciation a critical flaw that immediately breaks the illusion of natural speech. Combined with the somewhat flat prosody, the output sounds distinctly synthetic and flawed.
Output B:
Prosody and Intonation: The prosody is much natural and varied, with appropriate pitch con- tours and stress patterns that align well with the meaning of the text. It sounds like human speaker.
Pacing and Rhythm: The pacing is excellent, flowing smoothly and naturally without any awkward pauses or rushed segments. The rhythm is consistent and pleasant to listen to.
Articulation and Clarity: All words are articulated clearly and precisely. There are no noticeable mispromunciations. sibilance issues, or other artifacts.
Overall Naturalness: This output sounds highly natural and human-like. The combination good prosody, smooth pacing, and clear articulation results in a very convincing and pleasant listening ex- perience.
Conclusion: Output A: Output
System Output (Example):
("result": 1, "reason": "In the reasoning, Output A is judged clearly worse than Output B (robotic prosody, major mispronunciation) while Output B is described as highly natural and human-like The final scores reflect this ordering with A (4) < B (8.5), the conclusion is consistent with the analysis.")
Table 12 summarizes the results on CoT outputs for SpeechJudge-Eval across the three models. All three models exhibit very high internal consistency: Gemini-2.5-Flash. SpeechJudge-GRM (SFT), and SpeechJudge-GRu
(SFT+RL) achieve 97.9%, 97.6%, and 98.2% consistency, respectively. This indicates that, least the coarse level captured by this automatic check, the CoT analyses are not arbitrary narratives but align well with the final naturalness preference.
Human evaluation of CoT faithfulness. Logical consistency does not guarantee that the reason- ing is correct or grounded in the audio. To measure faithfulness and hallucination, we conduct human evaluation with experienced speech researchers (the background of these subjects are de- tailed in Appendix H.l). For each model and each sampled SpeechJudge-Eval pair, the experts examine the CoT and assign a 1-3 score on three dimensions (1) Prosody and Intonation, (2) Pac- ing and Rhythm, and (3) Articulation and Clarity-which match the dimensions specified in our CoT prompt (Table Th and used by the models in their explanations. A score means "highly sensible (e.g., the COT cites concrete audio details and the analysis is correct)", 2 means "partially
29

---

Published as a conference paper at ICLR 2026
sensible (e.g., the overall good/bad tendency is right but details coarse partially off)", and means "not sensible mostly hallucinatory"
The results reported in Table 13 On average, all three models obtain scores around 2.0, indi- cating that their CoT rationales are generally meaningful rather than dominated by hallucination. Importantly, SpeechJudge-GRie GRM does not lose CoT quality compared to the Gemini teacher (SFT) slightly im-
though initialized from Gemini-generated rationales: SpeechJudge- proves the average score to 2.03 + 0.34, and after RL, SpeechJudge-GRM (SFT+RL) further in- creases it to 2.17 + 0.12. The largest gain appears in the "Articulation and Clarity" dimension (2.40 vs. 2.10 for Gemini), suggesting that RLVR encourages the model focus more accurately on pronunciation errors and intelligibilityyrelate the human study suggests that the RI stage not only improves preference alignment but also mildly enhances the faithfulness of the CoT reasoning.
artifacts when explaining its decisions. Overall,
Differences in reasoning style. Finally, we examine whether the three models share the same reasoning style or develop distinct emphases. We randomly sample 20 SpeechJudge-Evar cases on which all three models predict the correct preference label, and collect their CoT outputs. We then submit these triplets of CoT's (anonymized as "Model 1/2/3") to three strong text LLMs-GPT 5.
Gemini Pro 15 and DeepSeek- Se asking them to compare the similarities and systematic differ- ences among the models. The specific instruction is as follows:
System Prompt:
I would like you to analyze the reasoning processes in the Chain-of-Thhoggee (CoT) outputs of the follow- ing three models (referred to as Model Model 2, and Model 3, respectively). What are the similarities and significant pattern differences among them? I will provide you with 20 cases, and under each case, the outputs of Model Model Model will given.
User Prompt (Example):
Case: 1/20
[Text] The rainbow serves metaphor for life's diversity just varying droplet sizes create different spectra, our unique experiences shape the breadth of our existence. The wider the band, the richer the experience.
[Model 1]
[Model 2] ...
[Model 3]
#+ +4988espseesss0e000000 Case: 2/20 www.s=asaaseeeeeeooooo..........
[Text] They have committed sending us the proposals address concerns the end of January.
[Model 1] ...
[Model 2]
[Model 3]
The three analyzers broadly agree on the following qualitative picture: all models follow similar structural template (prosody - pacing - articulation - overall naturalness, followed by a 1-10 score), but they differ in what they emphasize:
Gemini-2.5-Flash: Described as the most sophisticated at linking prosody to semantic meaning and discourse structure, often explaining why an intonation pattern distorts the intended message. JudechJJdge-GRMM (SFT); Viewed as emotionally and narratively focused, with stable format- ting and slightly more generous tone; it emphasizes human-likenes, and expressiveness but is somewhat less detailed on low-level signal artifacts.
(SFT+RL): Characterized as more critical and technically oriented: it pays more attention to mispronunciations, noise, and clarity, sometimes with blunter wording. Some analyzers note that its formatting is slightly less uniform than SpeechJudge-GRN (SFT), but its focus on error severity and technical correctness is stronger.
SpeechJudge-GGR
hatps://chatept.com//
http://isssiiioiiool..oooo
https://chat.deesece..om
30

---

Published as a conference paper at ICLR 2026
Taken together, these analyses suggest that SpeechJudge- GRM does not simply imitate the teacher's explanation style. Instead, SFT initializes a shared analytical framework, while the RLVR stage shifts the model toward rationales that more tightly coupled to human preferences and to concrete acoustic evidence (especially articulation and clarity), without sacrificing internal consistency. We will release the full set of CoT examples and meta-analyses in our open-source release to facilitate further research on CoT quality and reasoning in audio reward models.
G.3 ERROR ANALYSIS OF SPEECHJUDGE -GMM
Although SpeechJudge-GRe performs well on SpeechJudge-Evar it still disagrees with the human on subset of it. We manually inspected these ertors and found several recurring patterns.
many errors the human-preferred sample contains mild background noise but clearly more human-like prosody and articulation, while the alternative cleaner yet more robotic or over-smoothed. Annotators consistently favor the livelier sample, whereas GRM sometimes chooses the cleaner one, indicating that it can over-emphasize acoustic cleanness when the trade-off is subtle.
Over-weigh cleanness vs. liveliness.
Prosody-articulation trade-offs. Another frequent pattern is trade-off between expressive prosody and technical correctness. Humans often prefer speech with natural rhythm and intona- tion despite minor pronunciation issues, while GRM occasionally favors the perfectly articulated but flatter reading. These cases reveal that the relative weighting between prosody and articulation still imperfectly captured.
Extreme expressive styles. Errors also concentrate in highly expressive styles. For emotional speech with very high FO or strong emphasis, humans interpret the exaggerated prosody appro- priate for the style, but GRM sometimes penalizes it as "unnatural" and prefers neutral reading. For whispers, the lack of voicing makes prosody judgments difficult; GRM occasionally fails to distinguish the more fluent whisper when both samples sound degraded.
Very small preference gaps. A small number mistakes arise when both clips are high-quality and differ only in subtle cues mnicro-pauses, breathing, slight emphasis shifts). In these cases GRM's predictions are effectively close to random, which is unsurprising given the weak supervision signal.
In summary, SpeechJudge-GRM'ss errors concentrate on nuanced trade-offs (clean vs. lively, prosody vs. articulation) and on challenging expressive styles, suggesting future work on modeling recording conditions, style-aware priors, and more targeted training examples.
H HIGH-QUALITY SAMPLE SELECTION AND SPEECHJUDGE-GRM
BASED ON
H.1 DETAILS OF SUBJECTIVE EVALUATION
During the construction of SpeechJudge- -Dtaa we hired human labelers from data crowdsourcing company. To verify the effectiveness of our training for them and to ensure the high quality both the dataset and the resulting SpeechJudge-Gii GRM, the human subjects for the final sample selection and TTS post-training experiments (Section 5.3 and 5.4) were all experienced speech generation researchers. All these researchers had extensive audio backgrounds, with minimum of two years of experience in speech synthesis.
We randomly selected the subjective evaluation samples from both SeedTTS-Eval (Anastassiou ct al. 2024 17) and Amppion-TTS--A -va-Eval (Zhang et al. 20025b c 18 The evaluation set for each system Figure 6 consists of 70 samples, while the set for each system in Figure [ contains 100 samples. Each audio sample in these evaluations received at least three independent ratings. These subjec- tive evaluation results show that the annotation quality of SpeechJudge- Data largely aligns with the judgments of professional researchers.
http://wwwnnnnnaasssooaaa/ssasan Fval
31

---

Published as a conference paper at ICLR 2026
Table 14: Post-training of Qwen2.5-0.5B-TT. based on SpeechJudge.
Regular
Articulatory
Code-switching
Cross-lingual
Expressive
WER
SIM
WER
SIM
WER
SIM
WER
SIM
WER
SIM
WER
SIM
Qwen2.5-0.58-T7S
2.63
0.698
10.53
0.69
23.87
0.666
10.51
0.593
11.10
0.706
11.73
0.668
INTP
2,06
0.697
8.62
0.694
18.37
0.663
7.12
0.588
9.80
0.708
9.19
0.670
w/ Speechludge Data
2.12
0.698
8.92
0.678
19.01
0.657
7.72
0.583
9.97
0.707
9.55
0.664
SpeechJudge-GGRe GRM (offline)
2.31
0.698
7.83
0.681
15.36
0.662
7.84
0.593
9.72
0.709
8.51
0.668
Spw.chhadgg-GGRM (onlinc)
2.35
0.696
8:45
0.674
15.87
0.653
7.82
0,580
9.79
0.702
8.85
0.661
H.2 OBJECTIVE RESULTS
We present the objective results (WER and SIM) of the Qwen2.5-0.5B-TTS. post-training in Ta- ble [4] The results show that four post-training methods significantly improve the WER. This trend 1s similar to the subjective intelligibility results shown in Figure 6a
Regarding the SIM metric, both w/ INTP and w/ SpeechJudge- outperform the bascline model, while the other two methods show slight decline. However, the objective SIM results appear to be in slight conflict with the subjective speaker similarity results in Figure 6b For instance, in the subjective evaluation, w/ INTP actually shows decrease in speaker similarity (Win: 24.30%, Lose: 32.90%).
(offline) either match or slightly
Through follow-up interviews with the subjects who participated in subjective evaluation, gathered additional qualitative insights. Participants consistently reported that the synthesized sam- ples, both before and after post-training, demonstrated excellent speaker similarity, closely match- ing the reference speaker's timbre and style. In most cases, participants found it challenging to distinguish any significant differences in similarity, leading them prefer selecting "Tie". For ex- ample, in Figure [b] all four methods have the highest "Tie" proportion, each exceeding 40%. This demonstrates that post-training methods centered on naturalness (SpeechJudge-basee based) or intelligi- bility (INTP-based) are not yet fully aligned with speaker similarity, which requires further research into speaker similarity alignment.
32


Reviews:
[{"review_id": "hbiusPBV7k", "reviewer": "Reviewer_kfy3", "summary": "This work provides a human-annotated dataset for training automatic evaluators of TTS quality, and evaluates current automatic metrics against this dataset.  The paper also describes a new evaluator that is trained using this data by fine tuning with a standard evaluator followed by reinforcement learning on the dataset.  This new evaluator does improve other automatic metrics.", "questions": "P3 153: What motivated the choice of those six models?  Are they representative enough that evaluators trained on this data will have validity across new models?\n\nFigure 1 is not readable; please make the window span the entire screen.\n\nP4 189  Can you add details on evaluator recruitment  thatare missing in the main section and left to the appendix? This is crucial for validating the dataset for use in other languages than Mandarin and are not really an afterthought - having confidence in the annotations is a main paper issue since the primary product here is a dataset.\n\nP5 256: What is the score of a most-likely-label predictor?  I am assuming that the distribution of labels, particularly after subselecting for only FA data, would not be uniform.\n\nP7 372: Please expand SFT acronym here.\n\nP9 451: Section 5.4 - I think you need to state a limitation: a true test of the effectiveness of the evaluator for training would be human evaluation.  The proxies used are fine, but having this really improve MOS would demonstrate the effectiveness of the technique.\n\nAppendix C2: Why not use standard interrater agreement metrics like Fleiss’ Kappa or Krippendorf’s Alpha?\n\nIn general, there are a lot of very simple equations that are not necessary, and can be eliminated to provide additional space for other things.  Math that is not used to make a point is not useful in the main section of the paper.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Provides the community with a much needed resource for building automatic TTS evaluators that can be incorporated into the TTS training.\n2. The work is comprehensive in that it shows an entire pipeline with data collection, evaluation of existing models, building a new evaluation model, and then training a new TTS system on it.  The latter stages lend credibility to the primary resource - the dataset.", "weaknesses": "1. I have some concerns about the ability of any annotator to assess fine-grained naturalness in a second language (L2) - this ability will vary widely across annotators.  This is a significant challenge for anyone to collect this kind of data, but it does make me wonder more about  evaluation other the Mandarin, English, and code switched sets all being together, since we would expect increased differential agreement in L2 settings.    The paper would be stronger if it addressed this directly in the analysis.\n2. Similarly, code-switched speech is much more difficult for systems to generate (see, for example, CS-FLEURS https://arxiv.org/abs/2509.14161 https://www.isca-archive.org/interspeech_2025/yan25c_interspeech.pdf where they had to reject a significant subsample.  I think this should be addressed directly in the paper.  I think the data are there (in that raters did label errors, so this could be reported for ZH/EN/Codeswitched).  This is mostly a minor point compared to the first point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rWlB52FevX", "reviewer": "Reviewer_dToi", "summary": "This submission introduces SpeechJudge, a comprehensive suite addressing the critical gap in aligning speech synthesis models with human perceptions of naturalness. It comprises three core components: SpeechJudge-Data, a large-scale human feedback corpus with 99K speech pairs annotated for intelligibility and naturalness across diverse languages and styles; SpeechJudge-Eval, a benchmark for naturalness judgment that reveals existing metrics and AudioLLMs (e.g., Gemini-2.5-Flash) achieve less than 70% agreement with human judgments; and SpeechJudge-GRM, a generative reward model trained via a two-stage SFT+RL process on Qwen2.5-Omni-7B, which attains 77.2% accuracy (79.4% with inference-time scaling) and outperforms classic Bradley-Terry reward models. The work also demonstrates SpeechJudge-GRM’s utility in high-quality sample selection and TTS model post-training, with plans to release all resources to advance human-aligned speech synthesis research.", "questions": "How does SpeechJudge-GRM’s performance vary across specific cross-lingual settings (e.g., zh2en vs. en2zh) and expressive speech styles (e.g., emotional vs. accented), and what factors contribute to any observed differences?\nGiven that some open-source AudioLLMs show performance degradation with CoT prompts, did the authors explore alternative prompt engineering strategies (e.g., few-shot examples) for the SFT stage, and if so, what were the results?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Novel and Comprehensive Resource Creation: The construction of SpeechJudge-Data—with diverse TTS models, multilingual support, and dual annotations (intelligibility/naturalness)—fills a key void in large-scale naturalness-focused human feedback corpora for speech synthesis.\nRigorous Benchmark Design: SpeechJudge-Eval provides a standardized, high-quality evaluation framework that exposes limitations of existing metrics and AudioLLMs, offering clear direction for future improvements.\nEffective Reward Model Development: The two-stage SFT (with CoT rationales) and RL (on challenging cases) training for SpeechJudge-GRM is well-motivated, and its superior performance over baseline models, along with explainability and scaling capabilities, enhances its practical value.\nDemonstrated Real-World Utility: The experiments on sample selection and TTS post-training validate the framework’s applicability, showing tangible improvements in naturalness while preserving other key attributes like speaker similarity.", "weaknesses": "Limited Analysis of Cross-Lingual and Expressive Speech Performance: While the dataset includes cross-lingual and expressive samples, the paper lacks in-depth analysis of how SpeechJudge-GRM performs across these specific subsets, leaving uncertainty about its generalizability to diverse linguistic and stylistic scenarios.\nInsufficient Comparison with State-of-the-Art AudioLLM Judges: The evaluation of existing AudioLLMs focuses primarily on zero-shot performance with basic prompts; a more thorough comparison with fine-tuned or prompt-optimized AudioLLM judges (e.g., AudioJudge) would better contextualize SpeechJudge-GRM’s advancements.\nLack of Discussion on Annotation Bias: Although inter-annotator agreement is analyzed, there is no exploration of potential biases (e.g., linguistic, cultural) in human annotations, which could impact the reliability of the dataset and the model’s alignment with diverse human preferences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nhNJOdmlHH", "reviewer": "Reviewer_GQeA", "summary": "This paper introduces a corpus containing paired TTS samples with human preferences, a benchmark built upon this corpus for evaluating speech naturalness, and a reward model trained on it.", "questions": "1. It is interesting that even with paired comparisons—rather than direct scoring of individual samples—human judgments still exhibit a notable degree of disagreement. Could the authors provide any insights or analysis on the sources of this variability?\n2. It might be preferable to have native English speakers assess the naturalness of the English speech samples, especially given the high quality of recent TTS systems, where subtle perceptual nuances may be more easily detected by native listeners.\n3. Considering the observed inconsistency among human annotations, releasing the individual ratings (in addition to the majority-vote labels) could be valuable for future research, enabling studies on annotator variability and modelling of subjective perceptual uncertainty.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The collected large-scale human preference dataset can serve as a valuable resource for research on the automatic assessment of synthesized speech quality. The paper verifies its effectiveness both as a benchmark and as training data for building a model to automatically evaluate speech naturalness.", "weaknesses": "1. The corpus includes both regular and expressive samples, but the evaluation focuses solely on naturalness. Would it be possible to also consider expressiveness as an evaluation dimension, given that it is explicitly represented in the data?\n2. It is not clear why “tie” annotations are excluded from both the evaluation subset and the GRM training data. It might also be valuable for the model to recognize when two samples are of similar naturalness (i.e., no perceptible difference), as this could provide a more nuanced understanding of perceptual similarity.\n3. Gemini-2.5-Flash is used as the teacher model to generate CoT data for the first SFT stage. However, as shown in Table 2, its performance, while the best among compared models, is still suboptimal. Could the authors provide more insight into the quality of the CoT generated by Gemini 2.5? Was any further inspection or manual verification conducted to ensure its reliability?\n4. The paper mentions that during the RL stage, only the naturalness judgment is constrained, while the model is allowed to “autonomously optimize its reasoning and rationale generation capabilities.” I am curious about how is the model’s reasoning ability. Is there any quantitative measurement, qualitative analysis, or other inspection into the model outputs?\n5. The dataset already contains paired preference data, which, as acknowledged in the paper (Line 61), is well suited for DPO-style alignment. It might therefore be useful to include a DPO-based baseline to compare against, in order to assess how effectively the model can model human preferences.\n6. It might be informative to evaluate whether a model trained with SpeechJudge-GRM as the reward model can also predict absolute MOS scores. Since absolute MOS scores are generally more practical and efficient for automatic assessment and benchmarking of TTS quality than pairwise comparisons, such an analysis could further demonstrate the utility of the proposed data and model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vpk4bvqXiM", "reviewer": "Reviewer_KixZ", "summary": "This paper introduces SpeechJudge, a comprehensive framework for aligning speech generation models with human perceptual preferences. The suite includes:\nSpeechJudge-Data: a large-scale human preference dataset (99K speech pairs) annotated for intelligibility and naturalness;\nSpeechJudge-Eval: a benchmark for evaluating automatic speech naturalness judgment;\nSpeechJudge-GRM:a generative reward model trained on the dataset using SFT + RL (GRPO) to predict human preferences.\nExperiments show that existing MOS predictors and AudioLLMs (even Gemini-2.5-Flash) perform below 70% agreement with human judgments. The proposed GRM reaches 77.2% (79.4% with inference-time scaling), outperforming a Bradley-Terry reward baseline (72.7%). The model also improves the naturalness of post-trained TTS systems when used as a reward function.", "questions": "1. How were the expressive prompts and multilingual scripts quality-controlled?\n2. Did the authors analyze cases where the GRM and humans disagree—what aspects of speech cause errors?\n3. How generalizable is SpeechJudge-GRM to unseen voices (real human speech) or out-of-distribution languages (e.g., non-English/Chinese)?\n4. Could CoT reasoning introduce bias by inheriting the teacher model’s style of explanation?\n5. Did the authors analyze cases where both audios were judged “unnatural”? \n6. How do you ensure that the reward model is not overfitting to specific TTS model artifacts rather than perceptual cues?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Targeted an important open problem: aligning speech generation with perceptual human judgment.\nProvides a relatively large corpus (99K pairs) of human preference labels, which could be a useful resource for benchmarking.\nIntegrates SFT and RLHF (GRPO) techniques for audio reward modeling, which conceptually connects textual RLHF paradigms with audio generation.\nOffers comparisons across a broad set of AudioLLMs (Gemini, Qwen, Kimi, GPT-4o), giving some insight into current model limitations.", "weaknesses": "1. Ill-defined concept of “naturalness”\nThe central concept of speech naturalness is not operationally defined or quantitatively measured in an absolute or reproducible way. Annotators are instructed to make pairwise comparisons (“which one sounds more natural”), but the paper does not clarify whether they were asked to assess naturalness along specific perceptual dimensions such as prosody, timing, articulation, or spectral quality. As a result, the collected data are ordinal rather than metric, encoding only relative preference (i.e., which sample is less unnatural). This design becomes problematic in cases where both audios are of low quality—for instance, in the example on their project website (“case 1”), both clips mispronounce the word 24, leaving no meaningful basis for determining which is superior. Furthermore, the paper provides no evidence that annotators followed consistent perceptual criteria or were calibrated to established standards such as MOS or CMOS protocols.\n\n2. Dataset built entirely from synthetic TTS outputs. The authors explicitly state that all 99K samples come from synthetic TTS models (CosyVoice2, F5-TTS, MaskGCT, etc.) using synthetic prompts. This raises several issues: No human speech ground truth. The corpus lacks human-recorded references, so the model never learns what “human-like” actually sounds like. Model bias inheritance. Because all training data originate from a small set of TTS architectures, the reward model is likely to internalize architectural artifacts (e.g., vocoder noise, prosody style) rather than perceptual naturalness. Circular evaluation. The benchmark and reward model are both trained and tested on samples generated by the same model family, leading to evaluation leakage and poor generalization\n\n3. Limited analysis of annotation reliability and bias.\nWhile the paper discusses agreement levels (FA/WA/WD/FD), more detail is needed on labeler consistency, potential cultural bias in “naturalness,” and language-specific reliability. For a human-feedback dataset, these issues are central.\n\n4. Overreliance on closed models (Gemini-2.5-Flash).\nThe teacher model for generating CoT rationales is proprietary. The dependence on a non-reproducible component undermines the claim of full openness and may limit reproducibility of the SFT stage.\n\n5. Statistical analysis of results is thin.\nAccuracy gains (e.g., +4.5%) are reported without confidence intervals or significance tests. Given the subjective nature of the task, statistical reliability is important.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)