YuE: Scaling Open Foundation Models for Long-Form Music Generation
Authors: Ruibin Yuan
Hanfeng Lin
Shuyue Guo
Ge Zhang
Jiahao Pan
Yongyi Zang
Haohe Liu
Yiming Liang
Wenye Ma
Xingjian Du
Xeron Du
Zhen Ye
Tianyu Zheng
Zhengxuan Jiang
Yinghao Ma
Minghao Liu
Zeyue Tian
Ziya Zhou
Liumeng Xue
Xingwei Qu
Yizhi LI
Shangda Wu
Tianhao Shen
Ziyang Ma
Jun Zhan
Chunhui Wang
Yatian Wang
Xiaowei Chi
Xinyue Zhang
Zhenzhu Yang
XiangzhouWang
Shansong Liu
Lingrui Mei
Peng Li
Junjie Wang
Jianwei Yu
Guojian Pang
Xu Li
Zihao Wang
Xiaohuan Zhou
Lijun Yu
Emmanouil Benetos
Yong Chen
Chenghua Lin
Xie Chen
Gus Xia
Zhaoxiang Zhang
Chao Zhang
Wenhu Chen
Xinyu Zhou
Xipeng Qiu
Roger Dannenberg
Jiaheng Liu
Jian Yang
Wenhao Huang
Wei Xue
Xu Tan
Yike Guo
Venue: ICLR 2026 Poster (2026)
Keywords: lyrics2song
song generation
long-form
foundation model
music generation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We scale up an open LM-based song generation model to match the performance of proprietary systems.

Abstract: We tackle the task of long-form music generation, particularly the challenging \textbf{lyrics-to-song} problem, by introducing \textbf{YuE (乐)}, a family of open-source music generation foundation models. Specifically,
YuE scales to trillions of tokens and generates up to five minutes of music while maintaining lyrical alignment, coherent musical structure, and engaging vocal melodies with appropriate accompaniment. It achieves this through \textbf{track-decoupled next-token prediction} to overcome dense mixture signals, and \textbf{structural progressive conditioning} for long-context lyrical alignment. In addition, we redesign the \textbf{in-context learning} technique for music generation, enabling bidirectional content creation, style cloning, and improving musicality. Through extensive evaluation, we demonstrate that YuE matches or even surpasses some of the proprietary systems in musicality and vocal agility (as of 2025-01). We strongly encourage readers to \textbf{listen to our demo}\footnote{\url{https://map-yue.github.io/}}.

Published as a conference paper at ICLR 2026
YuE: SCALING OPEN FOUNDATION MODELS FOR LONG-FORM MUSIC GENERATION
Multimodal Art Projection', Hong Kong University of Science and Technology"
ABSTRACT
We tackle the task of long-form music generation, particularly the challenging lyrics-t-ssong problem, by introducing YuE (#), a family of open-source music generation foundation models. Specifically, YuE scales trillions of tokens and generates up to five minutes of music while maintaining lyrical alignment, coherent musical structure, and engaging vocal melodies with appropriate accompaniment. It achieves this through track-decoupled next-token prediction to overcome dense mixture signals, and structural progressive conditioning for long-context lyrical alignment. In addition, we redesign the in-context learning technique for music generation, enabling bidirectional content creation, style cloning, and improving musicality. Through extensive evaluation, we demonstrate that YuE matches or even surpasses some of the proprietary systems musicality and vocal agility (as of 2025-01). We strongly encourage readers to listen to our demd"
INTRODUCTION
Neural music generation is transforming artistic creativity and holds significant commercial and cultural potential (Ma et al., 2024). Among music generation tasks, lyrics-to-song creating full songs with vocals and accompaniment from lyrics, remains highly challenging. By the release time of our system, open- system can reliably achieve this scale, restricting innovation and accessibility despite proprietary solutions like Suno and Udia Jemonnssnntgiiingg promising results.
Key challenges of lyrics-to-song generation include: 1) Longerange dependencies: Music exhibits complex temporal structures spanning several minutes, making difficult for models to maintain coherence over extended durations. 2) Signal complexity: Unlike speech or environmental sounds, music is inherently polyphonic, requiring precise coordination between multiple instrumental and vocal components. 3) Linguistic distortion: Singing alters phonemes, durations, and prosody in ways that differ significantly from spoken language, complicating the alignment between lyrics and melody. 4) Data scarcity: The lack of large-scale, high-quality paired datasets of lyrics, vocals, and accompaniment limits model training and generalization capabilities.
Existing models for music generation and singing voice synthesis (SVS) are often limited in several key aspects. For instance, models such as Jukebox (Dhariwal et al., (020) and MusicLM (Agostinelli et al.) 2023) typically generate instrumental music with short durations (around 30 seconds) and lack coherent lyrical semantics when attempting to incorporate vocals. While some SVS systems (Chen et al. 2020; Liu et al. 2022) have been successful in generating vocals, they often rely explicit melodic guidance and do not seamlessly integrate vocals with instrumental accompaniments, especially over long-form durations. Moreover, academic models such as Me (Li al. 2024) and SongCreator (Lei al.) (0224) struggle to maintain high-quality, long-duration synthesis beyond 30 seconds, often suffering from artifacts or limited controllability. industry-develope systems, such Tiangong Music and Suno, offer better performance but remain proprietary, with limited technical transparency.
Related Work.
"Co-first affiliation. These institutions contributed equally.
https://map-yue. wht....thhhhh.... io/
https::/suno... com/ https::t//// www..wwwdd......o....o....o..... com/

---

Published as a conference paper at ICLR 2026
In this paper, introduce YuE, the firs open-source foundation model family for high-quee high-quality, full-song -oinng generation. Built upon LLaMA2 architechture (Touvron et al. 2023b] Zhang
et al. 2024) and trained on trillions of tokens, YuE generates coherent, syniclly-aaignee and musically interesting songs up to five minutes in length, matching or surpassing proprietary systems (Figure
Our main contributions include:
1) Track-Decoupled Next-Token Prediction (NTP): A dual-token strategy that separately models different audio tracks (vocals, accompaniment) at the frame level, resilient to challenging low vocal-to-accompaninee ratio scenarios like metal (Section 2.2
2) Structural Progressive Conditioning: A progressive conditioning strategy for long-form music generation, enabling song-level lyrics following and structure control (Section 2.3
3) Redesigned In-Context Learning (ICL) for Music: novel ICL framework enabling advanced style transfer, voice cloning, and bidirectional content creation (Section 2.4).
2 YUE
2.1 OVERVIEW
YuE is an autoregressive (AR) language model (I.M)-based framework tailored for lyrics-to-song generation. As depicted in Figure [ YuE comprises four main components: an audio tokenizer (with lightweight upsampler), text tokenizer, and two LMs. The audio tokenizer converts waveforms into discrete tokens using semantic-acouste fused approach. The Stage- 1 LM (Figure (2 is track-decoupled, trained on text tokens and semantic-rich bottom-level audio tokens (codebook-0 from Residual VQ-VAE), modeling lyrics-to-song generation as an AR NTP task. In Stage-2, a smaller LM predicts residual tokens from codebook-O) tokens to reconstruct audio. Both LMs follow the widely-adopted LLaMA2 architecture [Touvron et al., 2023a [Tam 2024). Next, we will introduce the key techniques for training the Stage- LM.
Vocal
Accompaniment
Detokenization & Upsampling
Stage Language Model
2.2
NEXT-TOKER PREDICTION
Rearrange
Challenges of Standard NTP. Popular LM-based ap-
proaches for modeling long residual vector quantization (RVQ) sequences typically adopt multi-stage design (Wang al. 2023a Agostinelli et al. 2023 Borsos et al. (202)) where the first stage commonly uses single codebook token to repre- sent each audio frame x Let x1.7 (I1, (2,, ...,,... ZT) represent a sequence of audio tokens, where each x corresponds one frame. In a standard NTP framework, we factorize the joint probability of x1.7 as:
Stage-1 Language Model
Audio
Text
Tokenizer
Tokenizer
Jazs
Fly me
Figure 1: Overview of YuE:
two-stage lyrics-to-song generation. Stage-1: music LM. Stage-2:
T
P(XET) = Il P(z (=1
x2c: x 8),
(1) residual modeling. Blue: vocal tokens. Orange: accompaniment
tokens. Grey; residual tokens.
where 0 is the model parameter. During inference (generation). the model predicts the next token di; which maximizes the conditional distribution:
S max P(zz xeti 0).
As of its release Jan. 28, 2025, YuE familiy is the first publicly available, open-source lyrics-to-sooe model capable of full-song generation with quality on par with commercial systems.
"We acknowledge angle-stage methods such as MusicGen, which utilize delay or parallel decoding patterns to reduce sequence length. However, observed that the parallel decoding pattern fails to converge on dataset, while the delay pattern results longer sequences compared multi-stage approaches.
2

---

Published as a conference paper at ICLR 2026
This approach works well for tokens X1:7 representing purely vocal (text-to-speech, TTS) instru- mental (text-to-music TTM) signals but struggles when encoding both vocals and accompaniment simultaneously due to differing dynamics, in lyrics-to-sone tasks combining TTS and TTM. We observed that RVQ compression causes significant linguistic information loss in genres with high acoustic complexity (e.g., metal). More discussions in Appendix
Track- Baack-Decoupled Next-Token Prediction (Dual-NTP). The above challenge shows that the issue arises from forcing a single token I₃ represent two distinct signals: vocal and music. Accompaniment can overshadow the vocal track, degrading lyric intelligibility. To overcome these limitations, we propose to explicitly use a source separation prior, splitting each time step into two tokens: one for vocal and one for accompaniment (see dotted token pairs in Figure 2
In the proposed method, each time step & outputs two tokens: U/ (vocal token) and a, (accompaniment token). The model's sequence of tokens thus becomes:
U1
42
VT
QT
vocal
accomp.
vocal
accomp.
vocal
accomp.
To formally define this, let VI.T (U1, U2, ..., UT) and a1.T (01, aa.....,ay... We factorize their joint probability as:
P(VIT, aIT)
II P Ut at veer acci )) t=1
Although this probability is written in joint form, it can be decomposed as:
P ct, at
Vet: acs; 0 = P("; vet, acti 0) x P P(at uie, acti 0),
(5)
making it straightforward implement in standard AR decoding frameworks.
Discussion. Prior work on dual-track modeling explores range of designs (Lei et al. 2024 Li
et al. 2024), but either requires substantial modifications to the base LM or treats the two tracks in sequential pipeline, incurring latency and error propagation. Our approach jointly models both tracks with minimal architectural changes and no sequential dependency, yielding these advantages.
1) Scalability: By preserving the existing LM architecture, we leverage well-established pre- training infrastructures and enable straightforward scalability.
2) Convergence:
Empirically, Dual-NTP converges to lower training loss compared to standard NTP (see Section Notably, it demonstrates robust lyric adherence even within challenging minority genres (e.g., metal music
3) Joint Modeling of Tracks:
Our approach jointly contextualizes both tracks a single forward pass, avoiding track synchronization issues, and allowing coherent and natural musical planning.
2.3 STRUCTURAL PROGRESSIVE CONDITIONING
Challenges of Full-song Generation. While typical TTS and TTM systems operate less than
30 seconds of context (Liu et al. 2023 2024b Wang et al. 2023a Borsos et al., 2023 Copet et al.
2023), full-song modeling requires handling minutes-long contexts. We find that extending the LM context to full-song modeling is non-trivial. The commonly used prefix text conditioning degrades as the length of audio tokens increases. Empirically, this degradation begins around 3K tokens and leads to complete failure beyond 6K tokens. Our mitigation attempts, such as increasing the RoPE base (from 10K 100K) or curriculum learning with gradually increasing audio lengths, have been ineffective. See ablation in Section for more details.
Structural Progressive Conditioning (SPC). To address this, we propose an elegant solution for Stage-1 LM that leverages the inherent structural priors of music. Songs are typically composed of distinct segments, such as intro, verse, chorus, bridge, and outro (Nicto et al. 2020 Bruderer et al.
2009 Lerdahl & Jackendoff 1996). We use all-in-one (Kim & Nam 2023) to automatically segment
'ee encourage the readers listen our demo.
3

---

Published as a conference paper at ICLR 2026
Same Timesleps
Misture Tokens
<SDA>
<DO>>
<80D>
Audio Tokens
Residual
E
Stage-1 Language Model
Lyrics2Song
Texi2Speech
Mix Music
Uncond.
E
Demix Music
Figure 2: The Stage-1 Framework of YuE. Dotted lines: Dual-NTP (Section 2.2
Text interleave:
SPC (Section 2.3).
Green tokens: ICL (Section 2.4)
songs into musical sections, with most of the sections shorter than 30 seconds. Within each structure section, text conditions (i.e. lyrics and structure tags) and audio are paired together. From a full song perspective, structured text and audio tokens are interleaved (see lyries2song token arrangement in Figure 2). In other words, each document in SPC begins with an instruction, tags, and raw lyrics, followed by series of interleaved lyrics-audio segments.
2.4 MUSIC IN-CONTEXT LEARNING
Deficiencies of Speech ICL. Previous work in TTS (Wang et al. 2023a Du et al. 2024) often defines speech ICL via continuation-based approach. The sequence constructed as:
Tref
Tipput
Arel
Agen
reference text
imput text
reference andio
generated audio
While this framework can be suitable for speech-based tasks, there are three major issues when directly applying it to music:
1) Necessity reference text. Requiring a text transcript for the reference audio can redundant in musical context, and lyrics may be unavailable challenging to obtain.
2) Unidirectional assumption. Continuation is unidirectional and restricts the task generalization in scenarios requiring bidirectional creativity, egg., writing an entire piece from short motif.
3) Entanglement. Continuation imposes strong constraints the style and content of the gener- ated audio. Given that music often features structural repetition, the model may simply replicate the reference melody or even entire segments, raising copyright concerns.
Re-designing ICL for Music. The aforementioned issues necessitate novel approach to ICL for music. We propose revised formulation of music ICL by extending the ICL format from SPC data. We randomly sample a 30s segment from the reference track and prepend its tokens to SPC data:
Diel Aret Dspe.
We find that this form of ICL can be effectively activated with minimal computational overhead (~22% of the total pre-training cost). However, ICL. constitutes strong conditioning signal and can be considered as "easy" data. Our preliminary experiments reveal that incorporating ICL data too early encourages shorteut learning (Geirhos et al.) 2020), where the model tends to directly copy the reference audio rather than composing novel music. This strong content entanglement even disrupts lyrical control. Once shortcut learning occurs, the model's creative capabilities cannot be easily restored. Removing ICL data and continuing training on SPC alone fails to resolve the issue- without reference audio, the model struggles to generate meaningful outputs, exhibiting poor musicality.
To address this, we introduce a delayed activation strategy. We introduce a small amount of ICL data (~B tokens) only during the annealing phase, ensuring ICL data is used beforehand. This
4

---

Published as a conference paper at ICLR 2026
strategy facilitates disentangled control between text and reference audio. For instance, using Japanese city pop track with a female vocal as a reference, the model can transform the lyrics into English while preserving the same vocalist and genre, even generate male rap version of it.
2.5 TOKENIZATION AND STAGE-2 2 LM
Tokenization. Our text vocabulary relies on the LLaMA tokenizer, which handles instructions, genres, and lyrics. We expand the vocabulary to support audio tokens. We use X-Codec (Ye et al. [024] as our audio tokenizer. It combines semantic and acoustic information in fused token, facilitating faster convergence. lightweight upsampling module is also applied. More details are Appendix C
Stage-2 Residual Modeling, Given Stage-1's 's tokens (codebook-O), Stage-2 uses an AR LM to jointly predict all
codebooks (0-7) on a strictly time-aligned stream, which first reads the entire codebook-O, then emits per-frame 8-tuples to align semantics with residual detail. The model trained with teacher forcing; at inference, codebook-0 is clamped to Stage-1 and only residual codebooks 1-7 generated, preserving alignment while refining audio. More details in Appendix
3 EXPERIMENTS
3.1 DETAILED SETUP
Pre-training. During pre-training, we employ multitask learning, including TTS, lyrics-to-song and unconditional music generation tasks jointly develop vocal and instrumental modeling. The final Stage-1 LM has 7B parameters, and the Stage-2 LM has 2B. 70k hours of speech and 650k hours of creative commons license music mined from the web used. Prior to annealing, the data mixture is set at Conditional Unconditional = =3 I and Music Speech = 10: 1. During annealing, only SPC and ICL data are used, maintaining ratio of SPC: ICL 2: 1.
Most Stage-1 experiments use a 0.5B model with a 100B token budget. For scaling, we increase the token budget 500B and scale models 0.5B, 2B, and 7B parameters. The 7B model is further trained on 1.75T tokens with 16K context, followed by a 40B-token annealing phase. Stage-2 experiments use 2T tokens and 8K context. We maintain a global batch size of 768. maximum LR of 3e-4 with linear warm up is applied, and anneals to 3e-5 during the annealing phase.
Test-time Strategies. Sampling and Classifier-Fres Guidance are applied to improve the good-case rate. Music ICL, which utilizes a song's chorus as prefix, enhances musicality and stability.
3.2 EVALUATION PROTOCOL
Baselines. We compare against four widely used proprietary music-generation systems, including Suno V4, Udio, Hailuo, and Tiangong which are representative of current production- grade models.
We conducted a human evaluation involving 40 researchers, including 12 experts in Speech/Music And 7 trained musicians. None of the evaluators participated in model training, ensuring objectivity. Following prior studies (Donahue et al. 2023 Qu et al. 2024 Yuan al. 2024), adopted A/B test format. See Appendix for detailed protocol.
Human Evaluation.
Automatic Evaluation. We also report automatic evaluation metrics, including Kullback-Leiblr (KL) divergence for measuring distributional differences in generated audio features using audioled dm__va. Frechet audio distance (FAD) (Kilgour et al. 2019) for assessing audio quality and realism (also audi dm_evallllllllmmmmmmmmmmmm Audiobox-Aesthttic ( (Tjandra et al. 2025) for capturing perceived musical aesthetics based on production quality (PQ), production complexity (PC), con- tent enjoyment (CE), and content usefulness (CU) using a neural audio embedding model, CLAP to measure semantic alignment between text prompts
score" and CLaMP score (Wu et al. 2025 10
https:/ /www https: wwww h cn/music
/https://suno com
https://hailuon con/music
Worked on htw.oo/eeooo-ppecccc.ooo-oo-oooo sing voice synthesis.
hiteer//gghuub.commmaaaaaaaaaaau eval
https://qithub. htteas//Attabb..ccoo////llll//// --
5

---

Published as a conference paper at ICLR 2026
YuE Wins
YuE Loses
Udio
19
53
24
70
51
Suno
Udio
14.0
Suno
16.3
18.6
Halluo
36
17
Hailuo
714
11.3
26.7
Tiangong
50
49
29
69
30
opp0
BuoBuell-
Tiangong
37.2
0%
25%
50%
75%
100%
Figure 3: Human evaluation comparing YuE to proprictary systems. YuE matches two of it (Tiangong, Udio) and outperforms one (Hailuo). Left: Average human preference on all aspects. Each cell (y, x) denotes the win rate of the system on the y-axis against the system on the x-axis. The leftmost "Overall" column shows the aggregate win rate against all other systems. Brighter colors/ larger numbers indicate higher preference. Right: win-tie-loss on musicality.
and audio outputs, vocal agility quantifying song-level vocal range and flexibility (pitch estimated with RMVPH applying 40ms note filtering and human verification), and generation duration practical measure of song-level audio modeling capability.
4
MAIN RESULTS
Human Evaluation. As shown in Fig. 3 YuE is competitive with four proprietary systems both average human preference" and musicality. It surpasses Hailuo by a clear margin, performs on par with Tiangong and Udio, but trails Suno V4 (the strongest system Focusing on musicality, YuF shows roughly balanced win-loss rates against Tiangong and Udio, decisively outperforms Hailuo, yet remains behind Suno V4. Overall, while proprietary models still lead, YuE narrows the gap and offers strong open-source alternative for music generation.
our comparison).
Vocal Agility. As shown in Fig. 4 the distribution of song-level vocal ranges across different systems reveals notable variations in vocal agility. Higher val- ues indicate greater vocal expressiveness. Among the models, YuE demonstrates one of the widest vocal ranges (medium ~= 27 semitones), closely match- ing top-performing closed-source systems like Suno
Distribution of Songelevel Ranges by System
Pange in Sentooves d0
%
V4. This suggests that YuE is capable of generating diverse and dynamic vocal performances. In contrast, models like Hailuo and Tiangong show more con- strained vocal range (a medium number around 20 semitones), indicating potential limitations in expres- siveness. These findings highlight YuE's strength producing vocally rich and varied song compositions.
Madk
Figure 4: Song-level vocal range on different systems. Higher values indicates better vocal agility, e.g. range=12 means the vocal only span through an octave a given song. YuE's vocal range is among the top close-source systems.
Duration. The distribution of generated song du- rations across different systems reveals substantial variation in length constraints as demonstrated by Fig.
5
YuE produces the longest audio, with significantly wider duration range compared all other models, demonstrating its ability to generate full-length songs beyond typical A Al-generated clips. Su and Tiangong also generate relatively long audio. In contrast, Hailuo Music show the most restricted durations, suggesting limitations in modeling long-term musical structure. These results
httpps: / / wwwpwwwwwwww..cceeoootceet coom/yllli htmm/yx111c//MM 1//MMVPF
"verraged across all aspects.

---

Published as a conference paper at ICLR 2026
WER vs VAR
vs Consumed Takens
Mixture WER
NTP
55
Mixture Recon WER
12-
00
Vocal Recon WER
WER (%6
50
AWER
45
Train
40
-80
-7.5
-7.0
-6.5
-6.0
-5.5
-5.0
4.5
VAR (dB)
0.
25
5.0
7.3
15.0
10.3
15.0
175
20.5
Figure 6: Comparison of WER-VAR plot for mixture and vocal tracks, including their tokenizer reconstructions, over 1K samples.
Consumed Train Tokens (Billion)
Figure 7: Training Loss over Consumed Train Tokens for NTP and Dual-NTP
highlight YuE's advantage in handling extended temporal dependencies, making it more suitable for full-song generation.
Duration Songs Generated by System
Model-based Evaluation. Table 1 shows model- based results, including distribution metries KL and FAD, aesthetics metrics, and audio-text alignment scores such as CLAP score (Wu et al., 2023) and CLaMP score (Wu et al. 2025). YuE outperforms others KL divergence (0.372) and performs com- petitively in FAD (1 (1.624), showing strong audio qual- ity and distribution matching. It also scores well on content-bac basen metrics, with scores close top sys- tems like Suno V4 and Tiangong, indicating strong audio aesthetics and usability. Regarding alignment metrics, YuE achieves the highest score with CLaMP 3 (0.240) but has a lower CLAP score (0.118). These discrepancies highlight potential limitations of the CLAP score in accurately capturing human per- ceptions controllability, possibly due differences in pretraining data and modeling strategies Note that not all metrics align well with human perception. A detailed discussion of correlation between each metric and human evaluation is provided in Appendix [H]
Daretion (sacondil
10p
Helua Mugi
usiia
Figure 5: Duration range different systems. YuE generates the longest audio.
Table I: Comparison of various music generation models across multiple metrics.
Metric
Distrib. Match
Content Based
Alignment
KL4
FAD4
CEf
CUt
PCT
PQ
CLAPt
CLaMP 3
Hailuo
0.756
2.080
7.350
7.737
6.793
8.132
0.265
0.106
SunoV4
0.620
1.544
7.474
7.813
6.601
8.120
0.265
0.160
Tiangong
0.708
2.547
7.421
7.766
6.060
8.220
0.244
0.114
Udio 0.503
1.222
7.112
7.520
6.626
7,803
0.310
0.156
YuE 0.372
1.624
7.115
7.543
6.280
7.894
0.118
0.240
ANALYSIS AND ABLATIONS
Effeet of Source Separation Prior and Dual-NTP. We define metric called the vocal-to- accompaniment Ratio (VAR), to quantify the effect of track-wise energy distribution on linguistic information loss. Let v(n) denote the vocal signal and a(n) denote the accompaniment signal, over
L'CLMMP3 3 is more recent model compared to CLAP, showing improved results in representation quality and music retrieval tasks due to extensive web-scale pretraining. In contrast, CLAAP may he limited by its exposure singing/music content during training, causing discrepancies in evaluating certain music types..
7

---

Published as a conference paper at ICLR 2026
..,,...,NN. ... We compute VAR (in dB) as follows:
VAR = 10 10 Σ (n) 10 Σ n=1 a²(n).
(6)
where higher VAR values indicate greater prominence of vocals relative to accompaniment, while lower VAR values suggest accompaniment dominance.
Figure 6 illustrates the WER- VAR relationship for mixture and vocal tracks across IK sam- ples, including tokenizer reconstructions. Though original vocal and mixture tracks exhibit similar absolute WER (solid blue and orange lines), mixture track reconstruction significantly increases WER (solid dotted blue lines), especially VAR declines, widening the gap (AWER). A 20%+ AWER is observed around -8.0 dB VAR. In contrast, vocal tracks main- tain low WER and smaller AWER (the worst case is 10%- around -8.0dB VAR), indicating resilience of source separation priors to VAR degradation and reconstruction information loss.
We also perform an ablation study comparing Dual-NTP and standard NTP. Figure 7 presents training loss curves of two 0.5B LMs trained with identical data and computational budgets (20B tokens). Dual-NTP shows a substantial reduction in loss (approximately 0.4 lower) com- pared to NTP, confirming its faster convergence. These analyses underscore the effectiveness of incorporating source separation priors with Dual-NTP into the song modeling task.
Word Error Rate Over Time
Word Error Rate (%)
100
80
60
SPC
A3" Sivass
Corfickin
M00 2.1g
20
30
60
120
150
Duration (seconds)
Effect of Lyrics-following Capabilities with SPC. The analysis in Figure 8 examines an ablation setting involving 0.5B LM, which was initially pretrained on a default mixture datase comprising 500B tokens and subsequently finetuned on the corresponding lyrics data for additional 200B tokens using the specified methods: Vanilla, Curriculum. and ABF, and our proposed SPC. Additionally, we include results from the YuE- 7B checkpoint illustrate the performance gains achievable through scaling. Specifically, Vanilla refers to text prepend conditioning, where the model is trained with prepended lyrics as input for conditioning. Curriculum involves gradually increasing the text prepend data with progressively longer durations (e.g.. 30s, 60s, 90s, etc.), aiming to improve the model's ability to follow lyrics over time. ABF (Xiong et al., 2023) refers to adjusting the rope base frequency from 10k to 100k during finetuning to explore its effect on lyrics-following performance. In Figure 3 the WER over time is estimated using a fine-tuned Whisper model, with measurements recorded every 30 seconds up to 150 seconds. Overall, the proposed SPC method achieves consistently superior performance across all evaluated time intervals (30s to 150s). Scaling the model to 7B parameters demonstrates substantial improvements, reducing the WER from approximately 70% at 0.5B parameters around 20% In contrast, Vanilla, Curriculum, and ABF methods exhibit substantially worse WER, indicating limited capability maintaining lyrical coherence. Through manual inspection, we identified that the primary reason for failure in Vanilla and Curriculum was their tendency to generate instrumental preludes, causing the onset of singing to drift far from the original prepended lyries condition.
Figure 8: WER over time. Both SPC and model scaling enhance lorics-following capability.
Effect of Scaling, We investigate the impact of model scaling on musicality and criis-folloowng capabilities. We compared checkpoints at 0.5B, 2B, and 7B scales. While the 0.5B and 2B models were trained with limited budget of 500B tokens (in 16K context), the 7B model underwent complete scaling with a significantly larger 1. 7.75T token budget using the full training dataset. In Figure (aa human evaluation demonstrates a clear improvement trend in both musicality and lyrics-following as model scale and training budget increase. Notably, the 7B model exhibits substantial enhancements, indicating that increased parameter counts and extensive training significantly boost the model's foundational creativity and compositional quality. These results confirm that scaling plays a crucial role in achieving higher musicality and improved lyric adherence.
A mixture of speech and music. Text transcripts are in prepend format. Note that 20% can be considered relatively low number. Refer the GT WER-to-VAL plot in Figure
8

---

Published as a conference paper at ICLR 2026
Win Rate Comparison
Rate Musicalty
Average Win Rates of Different Test lest-time tricks
Rate LyticsFolow
0.7
0.
2.5
0.0
0.43
02
0.37
0:21
0.38
Model Parameters
SPC
SPC
Settings
(a) Win rates for Musicality I yrics--ollowing across model scales in pairwise A/B tests.
(b) Win rates for musicality.
Figure 9: Win Rate Statistics.
Effect of Test-time Tricks. Figure presents human preference win rates for musicality obtained through A/B testing across different inference settings using YuE-7B checkpoints. Results clearly demonstrate that ICL-based methods outperform SPC-based methods significantly: ICL achieves a win rate of 0.63 compared to only 0.21 for SPC. Incorporating CFG further enhances these methods; specifically, ICL+CFG obtains the highest win rate (0.79) substantially exceeding both ICL. alone and the SPC-based configurations. This performance advantage stems from the strong conditioning ability of ICL, which restricts the decoded token space to a musically favorable subspace guided by the provided human generated music prompt. CFG similarly strengthens this conditioning by amplifying the influence of the text condition on next-token logits, making generated outputs more closely aligned with the intended prompt-guided subspace and thus further improving musicality.
numan-generated
Memorization Test. To assess if YuE's ICI. mode (30 audio prompt lyrics) reproduces training material, compute melody-aware cosine similarity with ByteCover2 (Du et al.) 2022) between = matched Ref (train- ing) Gen (ICL outputs) pairs, analysing the top-1% 1% tail in prior work (Agostinelli al.
Sinilarity Distilattin Compasioon
1101
0.76
0.50
2023 Yuan et al., 2024). For context, we report the metric GTZAN (genre-level pairs) and Covers80 (known duplicates). As shown in Fig. 10. Ref-Gen similarities are far below Covers80 and comparable to GTZAN. While occasional short motifs (e.g., pereussive loops) appear, we find no evidence of wholesale copy- ing; YuE primarily recombines learned patterns to produce novel content.
0.25
0.00
Bad--Gam
GIZAN
Figure 10: Box-plot comparison of cosine similarity across three scenarios: Covers80, Ref- (our training vs. generated sets), and GTZAN. The black bar denotes the median, and the diamond denotes the mean.
Emergent Abilities. Scaling YuE. substantially improves generation quality and reveals emergent behaviors: advanced vocal techniques (c.g.. vibrato, bel canto, death growl), spontaneous musicality (scat continuation, cappella harmonies. instrumental fills), global style fusion, and code switching. The model also achieves realistic singing voice cloning and versatile style transfer across languages. genres, and timbres while preserving lyrical/melodic structure and prosody, illustrated our demd 16
CONCLUSION
We introduced YuE, an open-source foundation model family for long-form lyrics-to-song generation. Leveraging large-scale data, Dual-NTP, SPC, and redesigned ICL framework. YuE generates coher- ent, expressive songs with detailed musical structure. Experiments demonstrate that YuE matches or exceeds commercial systems musicality, controllability, and lingull lyries following, and achieves competitive results on standard music understanding benchmarks.
"tttps://map-yur thub.io//

---

Published as a conference paper at ICLR 2026
CONTRIBUTIONS AND ACKNOWLEDGMENTS
The research was supported in part by Early Career Scheme (ECS-HKUUT1222012)) Theme-based Research Scheme under Grant T45-205/21-N from Hong Kong RGC, and Generative AI Research and Development Centre from InnoHK.
Core Contributors
Ziyang Ma, Eval MAP, SJTU, NTU
Ruibin Yuan, Lead, Pre-train, Data, Eval HKUST, Moonshot.ai, MAP, ryuanab @ connect.ust.
Jun Zhan, Eval
Fudan University
Hanfeng Lin, Pre-train, Data, Eval, Inference HKUST, MAP, unst.nll@ust.hk
Chunhui Wang, Eval, Pre-train
Geely
Shuyue Guo, Pre-train, Demo MAP
Yatian Wang, Eval
HKUST
Xiaowei Chi, Eval
Ge Zhang, Pre-train MAP, gezhang trchang@umich.ede udiicheduu
HKUST
Xinyue Zhang, Eval
Jiahao Pan, Pre-train, Eval, Data
HKUST
HKUST, MAP, fengshichcic gaan gomil..com
Zhenzhu Yang, Eval
HKUST
Xiangzhou Wang, Eval
MAP
Contributors
Shansong Liu, Eval
Yongyi Zang, Upsampler, Eval Independent
Meituan
Lingrui Mei, Eval
Haohe Liu, Upsampler. Tokenizer, Demo University Of Surrey, MAP
Meituan
Peng Li, Eval
Yiming Liang, Eval Lead
HKUST
MAP
Junjie Wang, Eval
Wenye Ma, Representation Learning
Tsinghua University
MBZUAI, MAP
Jianwei Yu, Data, Inference
Xingjian Du, Memorization Effect
Moonshot.af
University of Rochester. MAP
Guojian Pang. Inference
Xinrun Du, Pre-train
MAP
MAP
Xu Li, Eval
Zhen Ye, Tokenizer
Xiaohongshu
HKUST
Zihao Wang, Data
Tianyu Zheng, Pre-train
Zhejiong University, Carnegie Mellon University
MAP
Zhengxuan Jiang, Inference
MAP
Academic Advisors
Yinghao Ma, Eval
Xiaohuan Zhou
MAP, Queen Mary University of London
Minghao Liu, Eval, Data
MAP
Lijun Yu
2077A1, MAP
Carnegie Mellon University
Zeyue Tian, Eval
Emmanouil Benetos
HKUST, MAP
Queen Mary University of London, MAP
Ziya Zhou, Eval, Data
Yong Chen
HKUST, MAP
Geely
Liumeng Xue, Eval, Data
Chenghua Lin University of Manchester, MAP
HKUST. MAP
Xingwei Qu, Pre-train, Eval
Xie Chen Shanghai Jiao Tong University
MAP
Yizhi Li, Eval
Gus Xia
MAP, University of Manchester
Shangda Wu, Eval
MBZUAL MAP
Zhaoxiang Zhang
Central Conservatory Muxic, MAP
Chinese Academy Sciences
Tianhao Shen, Eval, Inference
Chao Zhang
MAP
10

---

Published as a conference paper at ICLR 2026
Tsinghua University
Jian Yang
Wenhu Chen
MAP. MaP.jaaya @ buau. buaa.edu.cn
University of Waterloo, MAP
Xinyu Zhou
Wenhao Huang
Moonshot.ai
MAP, ruhio8741 @ gmail.com
Xipeng Qiu
Fudan University
Roger Dannenberg
Wei Xue
Carnegie Mellon University, MAP
HKUST, weixue @ ust.hk
Xu Tan
Correspondence (Alphabetical Order)
Moonshot.ai, MAP, tanxu2012@ @gaiil
Jiaheng Liu
Nanjing University, MAP, 13121221227@ 16e0e/66366.. 163.com
Yike Guo
HKUST, yikeguo ust.hk
ETHICS STATEMENT
The authors have read and adhered to the ICLR Code of Ethics. We have taken proactive steps to address ethical considerations throughout this research, including responsible data sourcing under Creative Commons licenses, analysis of model fairness and bias across different languages and genres, and mitigation strategies for potential misuse, such as the generation of "deepfake music." " Our source release includes binding use terms to prevent misuse. This study was reviewed and approved by the Human and Artefacts Research Ethics Committee under protocol HREP--003-0223 A comprehensive discussion of our ethical considerations, including data handling, bias analysis, and risk mitigation, is provided in the Appendix
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our research, we has released our source code on GitHub and pre- trained model checkpoints on HaggingFace The Appendix [ad furtheer details the tokenization and model architecture, while Section providees hyperparameters and the computational environment required to replicate experiments. We are confident that these resources will enable the research community to verify our findings and build upon our work.
REFERENCES
Andrea Agostinelli, Timo Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. MusicL.M: Generating music from text. arXiv 2rep 2023.
Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek. Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. Au- diolm: A language modeling approach to audio generation. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 11:2523-25333 2023. doi: 10. 1..1109/AAS..220333288888
Michael J Bruderer, Martin F McKinney, and Armin Kohlrausch. The perception of structural boundaries in melody lines of western popular music. Musicae Scientiae, 13(2):273-313, 2009.
Jiawei Chen, Xu Tan, Jian Luan, Tao Qin, and Tie- Yan Liu. Hifisinger: Towards high-fidelity neural singing voice synthesis. arXiv prpp 2020.
Yuk

---

Published as a conference paper at ICLR 2026
Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. Simple and controllable music generation. arXiv preprint arXiv:2306.05228 00244, 2023.
Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. arXiv preprini arXiv:2210. 2022.
Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint wrXiv:2410.000533 2024.
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: generative model for music. arXiv preprint arXiv:2005.00044/ 2020.
Chris Donahue, Antoine Caillon, Adam Roberts, Ethan Manilow, Philippe Esling, Andrea Agostinelli. Mauro Verzetti, Ian Simon, Olivier Pietquin, Neil Zeghidour, et al. Singsong: Generating musical accompaniments from singing. arXiv preprint arXiv:2301. /266,, 2023.
Xingjian Du, Ke Chen, Zijie Wang, Bilei Zhu, and Zejun Ma. Bytecover2: Towards dimensionality reduction of latent embedding for efficient cover song identification. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 616-620. IEEE, 2022.
Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based supervised semantic tokens. arXiv preprint arXiv:2407 240.000077. 2024.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 1):665 0663666666666666 2020.
Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. Cnn architectures for large-scale audio classification. In 2017 ieee international conference on acoustics, speech and signal processing (icassp), pp. 131-135. IEEE, 2017.
Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fréchet audio distance: A reference-free metric for evaluating music enhancement algorithms. In Proc. Interspeech, 2019.
Taejun Kim and Juhan Nam. All-in-one metrical and functional structure analysis with neighborhood attentions on demixed audio. In 2023 IEEE Workshop Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 1-5. IEFE, 2023.
Khaled Koutini, Jan Schlüter, Hamid Egghal-Zadehh, and Gerhard Widmer. Efficient training of audio transformers with patchout. arXiv preprint arXiv:2/ 10.05069, 2021.
Shun Lei, Yixuan Zhou, Boshi Tang, Max WY Lam, Feng Liu, Hangyu Liu, Jingcheng Wu, Shiyin Kang, Zhiyong Wu, and Helen Meng. Songereator: Lyrics-based universal song generation. arXiv preprint ar (rXiv:240900600
2024.
Fred Lerdahl and Ray Jackendoff. Generative Theory of Tonal Music, reissue, with new preface. MIT press, 1996.
Ruiqi Li, Zhiqing Hong, Yongqi Wang, Lichao Zhang, Rongjie Huang, Siqi Zheng, and Zhou Zhao. Accompanied singing voice synthesis with fully text-controlled melody. arXiv preprint arXiv:2407.02049, 2024.
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. AudioLDM: Text-to-audio generation with latent diffusion models. Proceedings of the International Conference Machine Learning, 2023.
Haohe Liu, Xuenan Xu, Yi Yuan, Mengyue Wu, Wenwu Wang, and Mark D. Plumbley. SemantiCodec: An ultra low bitrate semantic audio codec for general sound. JEEE Journal of Selected Topics in Signal Processing, 1848):1448-144 1461, 2024a. doi: 10. 10.100995S99922224336266
12

---

Published as a conference paper at ICLR 2026
Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang. Wenwu Wang, Yuxuan Wang, and Mark D. Plumbley. AudioLDM 2: Learning holistic audio generation with self-supervised pretraining. IEEE/ACM Transactions Audio, Speech, and Language Processing, 32:2871-2883, 2024b. doi: 10.1 N0.1109/AASL.22243339999
Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, and Zhou Zhao. Diffsinger: Singing voice synthesis via shallow diffusion mechanism. In Proceedings of the AAAI conference on artificial intelligence, 2022.
Yinghao Ma, Anders Øland, Anton Ragni, Bleiz MacSen Del Sette, Charalampos Saitis, Chris Donahue, Chenghua Lin, Christos Plachouras, Emmanouil Benetos, Elona Shatri, et al. Foundation models for music: A survey. arXiv preprint arXiv:2408. 14340, 2024.
Oriol Nieto, Gautham Mysore, Cheng-i Wang, Jordan BL Smith, Jan Schlüter, Thomas Grill, and Brian McFee. Audio-based music structure analysis: Current trends, open challenges, and applications. Transactions of the International Society for Music Information Retrieval, 3(1), 2020.
Xingwei Qu Yuelin Bai, Yinghao Ma, Ziya Zhou, Ka Man Lo, Jiaheng Liu, Ruibin Yuan, Lejun Min, Xucling Liu, Tianyu Zhang. et al. Mupt: A generative symbolic music pretrained transformer. arXiv preprint arXiv:2404.0633) 2024.
Hubert Siuzdak. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. arXiv preprint arXiv:2306.0000 00814, 2023.
The Llama Team. Introducing meta Ilama The most capable openly available I1m date, 2024.
URL httph: //as:://ai. mwwa. com/ www.ccommolll.. www.b/log////eeee 1llmallm 37
Andros Tjandra, Yi-Chiao Wu, Baishan Guo, John Hoffman,, Brian Ellis, Apoorv Vyas, Bowen Shi, Sanyuan Chen, Matt Le, Nick Zacharov, al. Meta audiobox aestheties: Unified automatic quality assessment for speech, music, and sound. arXiv preprint arXiv:2502.051.3 2025.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. /3971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXin:2337.002888 2023b.
Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. Neural codec language models are zero-shot text to speech synthesizers. arXiv, abs/2301.02 10101.02111. 2023a.
Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, al. Neural codec language models are zero-shot text to speech synthesizers. arXiv 2reprimt:2001.02111 2023b.
Shangda Wu, Zhancheng Guo, Ruibin Yuan, Junyan Jiang, Seungheon Doh, Gus Xia, Juhan Nam, Xiaobing Li, Feng Yu, and Maosong Sun. Clamp 3: Universal music information retrieval across unaligned modalities and unseen languages, 2025. 10362]
Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Kirg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Wenhan Xiong, Jingyu Liu, Igor Molybog. Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. arXiv preprint arXiv:2309.1660 2023.
Dongchao Yang, Songxiang Liu, Rongjie Huang. Jinchuan Tian, Chao Weng, and Yuexian Zou. Hifi-codec: Group-residue vector quantization for high fidelity audio codec. arXiv preprint 02520530 2023a.
13

---

Published as a conference paper at ICLR 2026
Dongchao Yang. Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Xixin Wu, et al. Uniaudio: An audio foundation model toward universal audio generation. arXiv preprint arXhv:2310.00704, 2023b.
Zhen Ye, Peiwen Sun, Jiahe Lei, Hongzhan Lin, Xu Tan, Zheqi Dai, Qiuqiang Kong, Jianyi Chen, Jiahao Pan, Qifeng Liu, et al. Codec does matter: Exploring the semantic shortcoming of codec for audio language model. arXiv preprint 1711v:2408.17175. 2024.
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, et al. Chatmusician: Understanding and generating music intrinsically with Ilm. arXiv preprint arXiv:2402. 16153, 2024.
Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, et al. Map-neo: Highly capable and transparent bilingual large language model series. arXiv preprint 2rXiv:2405.19322 2024.
Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. Speechtokenizer: Unified speech tokenizer for speech large language models. arXiv preprint arXiv:2308. /6692, 2023.
14

---

Published as a conference paper at ICLR 2026
A
THE USE OF LARGE LANGUAGE MODELS (LLMS)
During the preparation of this manuscript, we utilized Large Language Model (LLM) as a general- purpose writing assistant. role was confined to proofreading for grammatical errors, correcting typos, and improving sentence structure for better readability. The LLM did not contribute to the core research ideas, experimental methodology, or the substantive content the paper.
B ETHICS AND RESPONSIBILITY
Data Sources and Licensing. We committed to responsible data collection. The primary sources for our training data are as follows:
The audio tracks were sourced from an online streaming platform. To respect creators' rights, specifically filtered for and downloaded content licensed under Creative Commons (CC), which permits reuse.
Audio Data:
Lyries Data:
The corresponding lyries for the audio tracks were programmatically retrieved using Google Search.
Opt-Out and Filtering: During data collection, we implemented a string matching filtering process to automatically exclude any data that carried an explicit copyright declaration or restrictive terms of use. This serves as our primary opt-out mechanism.
While we have taken these steps, we acknowledge the inherent challenges in verifying the copyright status for every item in web-scale dataset.
Language Distribution. The language of the lyrics estimated using external language identification model. The distribution skewed towards high-resource languages, with English being the most prevalent.
Table 2: Language distribution of lyrics (estimated).
Language ID)
Percentage
en 60.68%
zh 22.83%
es
6.07%
fr
1.49%
it
1.11% 1%1
ja
1.00%
ru
0.98%
pt 0.96%
de 0.89%
0.72%
Genre Distribution.
Top genre labels were extracted from available metadata.
Table 3: Genre distribution.
Genre
Percentage
Pop 19.1%
Rock
10.9%
Electronic
8.1%
Classical
7.0%
R&B
4.7%
Rap
2.3%
15

---

Published as a conference paper at ICLR 2026
Artist Demographies. We acknowledge that we do not have structured data artist demographics. This information was not consistently available in the metadata of our data sources.
Evaluation Minority Languages and Genres. We evaluate model's performance underrepresented groups. We curated a new test set of 1,000 world music tracks from diverse global cultures (mostly folk/traditional music from a corresponding area). We evaluated our model, YuE-7B, on this dataset in single-trace track mode and report the perplexity (PPL) for each language below. A lower PPL indicates better performance.
The results demonstrate that our model is competent across wide array of languages, many of which minorities our training data.
Table 4: Per-language perplexity the minority- test set (lower is better).
Language
Mean PPL
Language
Mean PPL
fr
25.53
39.12 19.12
ja
27.61
vi
39.16
da
28.16
bg
39.31
sw
28.49
hi
39.57
30.00
40.38
30.87
de
41.14
bn
31.09
uk 42.03
31.74
en 42.87
pl
31.84
it 42.89
th
32.41
43.72
zh
33.00
ko 44.68
id
33.15
iw 45.00
SV
33,39
sk
46.32
33,52
no
47.79
hu 34,05
ru
48,80
34.60
53.63
fi 34.86
es
35.15
hr 35.82
el 36.02
nl 38.51
38.66
Notably, the model achieves relatively better performance (low PPL) on languages like French (fr), Japanese (ja), and Swahili (sw), demonstrating promising ability to generalize beyond the majority languages.
Plagiarism. Ensuring ethical and responsible Al-generated music is crucial for fostering trans- parency, accessibility, and fair contribution to the music industry. As suggested by Ma et al.) (2024), to promote accountability, we advocate for the inclusion of Al-generated Al-assisted tags in generated content, increasing transparency for both musicians and audiences. Additionally, our memorization- effect experiments in Section deemontttete that our design maintains creativity without plagiarizing. even under strong training set conditioning.
In contrast to closed- source commercial systems, our model leverages an exceptionally diverse training dataset, explicitly enriched with culturally diverse music content. This enables the model to innovate and create within niche musical styles effectively (see demos). As such, our model can serve as a parameterized knowledge base, contributing to the preservation and expansion of human musical artistry and cultural heritage.
Risks and Mitigation. Despite these positive contributions, recognize that high-fidelit vocal and stylistic imitation enabled by YuE may inadvertently facilitate deepfake music generation, including unauthorized emulation of real artists' voices, styles, or lyrical identities. Such misuse
16

---

Published as a conference paper at ICLR 2026
poses ethical concerns regarding consent, authenticity, and potential economic harm to original creators. To mitigate these risks, we propose several precautionary measures for the following work: (1) integrating imperceptible audio watermarks into generated content to trace model origin, (2) establishing clear terms of use and content attribution guidelines in downstream applications, and (3) encouraging community-driven audits of model outputs for misuse detection. We believe that proactive govemance--looneeeeee transparency, open research practices, and active engagement with stakeholders-cant help balance creative empowerment with ethical responsibility.
Current Status. Our open-source repository already enforces binding use terms requiring: 1) Clear labeling of Al-generated content, 2) Compliance with copyright laws, 3) Prohibition of unauthorized voice/style impersonation. 4) We are developing watermarking tools for the model.
Ethics Committee Approval. This study has been reviewed and approved by the Human and Artefacts Research Ethics Committee under protocol HREP-2023-0230. The approval ensures that our research adheres to ethical guidelines in data usage, Al generation, and cultural representation. The approval remains effective until 30-Jan-2027
17

---

Published as a conference paper at ICLR 2026
C TOKENIZATION AND AUDIO RECONSTRUCTION
C.I OVERVIEW
Following the design space of Borsos et al. (2023); Wang et al. (2023b), the stage-1 1 LM models text tokens and semantic-rich codebook-O) tokens. After investigation, we realized that the vanilla text-to-speech (TTS) / text-to-music (TTM) method performs poorly on our task, where musicality and song-level lyrics--olllee following capability are the two key challenges.
Text Tokenizer. In this work, the vocab- ulary of the LMs contains two sections: text and audio. For the text part, we reuse LLaMA tokenizer with size of 32000 unique BPE tokens. Instructions, genres, lyrics, structure annotations, and structure segment boundary signals are represented with text format and tokenized with BPE.
Table 5: Special tokens and their descriptions.
Token
Description
<EOD>
End of document
<SOA>
Start of audio
<EOA>
End of audio
<stage.ll 1>
Start of Stage 1
<stage
Start of Stage
<encodec32k>
Tokenizer type (Encodec 32k)
Semantic-Acoustie Fused Codec. For the audio vocabulary. we experimented with several open-source music and univer- sal neural codecs. Ultimately, we adopted semantic-acuustic fused strategy (Défossez) ct al. 2024 Zhang et al. 2023 Liu et al. 2024a Ye et al. 2024). Specifically, we utilized X-Codec Ye et al. (024) as our off off-the-shelf audio tokenizer. We employed cenpral-purpose version of X-Codec, trained on a mixture of 200k hours 16 kHz audio with ratio of music speech : audio
<xcoaec>
Tokenizer type (XCodec)
<semanti codec>
Tokenizer type (SemantiCodec)
<hifi
Tokenizer type (HiFiCodce)
effects 1: 1: 0.05.
The X-Codec tokenizer fuses a 100M-parameter HuBERT- based universal semantic representation into the codec latent space. It has 50Hz frame rate, consists of 12 RVQ layers, each with codebook size of 1024. For this study, we used only the first 8 layers, as including more layers did not yield noticeable quality improvements. Notably, codebook-0 alone captures rich semantic information such as melody and vocal content, which are critical for our task.
Vocabulary Expansion and Special Tokens. We expand the SentencePiece tok- enizer vocabulary to support multiple audio tokenizers and special tokens. Specifi- cally, include Encodec-32kh2 32uhh- (Défossez et al., 2022 Copet al. 2023)
HiFi-Cotec-universal (Yang et al., 2023a b). X-Codec-generaa
Ye et al. 2024). and
crrantiicodee1100tps (Liu al. 2024a).
For special tokens, we introduce the following: <EOD> represents the end of a document, <SCA> denotes the start of audio, and <ECA> signifies the end of audio. Additionally, stage indicators, <stage_1> and <stage_2>, mark the beginning of Stage I and Stage 2 tokens, respectively. Tokenizer type indicators specifyy the corresponding tokenizer types, which are inserted between <SCA> and the actual audio token IDs. Note that stage indicators are only used in residual modeling and positioned between <SOA> and the tokenizer type indicator.
Light Upsampling Module. To achieve better perceptual audio quality, we upsample the reconstructed 16kHz audio to 44. 1kZ.. For this, we utilize weight upsampling vocoder adapt- ing Vocos (Siuzdak] 2023) to predict the higher-frequeng components. To enhance the robustness of the upsampler, we apply codebook dropout randomly and introduce a small amount of Gaussian noise during training.
C.2 COMPARISON OF AUDIO TOKENIZERS
In preliminary experiments on a 130k-hour subset of diverse music data, we conducted qualitative analysis of four popular audio tokenizers, specifically focusing acoustic tokens and fused semantic- acoustic tokens (see 6). Separate semantic and acoustic tokenizers would require retraining and thus were beyond the scope of this study, reserved for future work.
18

---

Published as a conference paper at ICLR 2026
Table Qualitative comparison of different codec types based reconstruction quality, LM convergence, and invalid probability. Invalid probability refers to the likelihood of generating noise or silence segments during LM token synthesis.
Type
Codec
Reconstruction
LM Converge
Invalid Prob.
Acoustic
Encodce32k HiFiCodec Semanticodec
Good
No
All
Acoustic
Good
No
All
Semantic Acoustic
Fair
Yes
High
Semantic Acoustic
X-Codec
Fair
Yes
Low
Acoustic tokenizers, including Encodec32k and HiFiCodec, exhibited decent reconstruction quality. However, their learned tokens proved challenging for LMs converge due to the complexity and variability inherent in our in-the-wil wild dataset. Training a 0.5B LM with acoustic tokens consistently failed to converge, resulting primarily in invalid outputs characterized by noise or silence. Although prior studies indicated Encodec32K has been successfully applied to TTM (Copet et al. (023), even scaling the LM to 7B and extending training up to trillion tokens on our data yielded only intermittent success, with outputs still dominated by noise.
In contrast, tokenizers integrating semantic and acoustic features (Semanticodec, X-Codec) demon- strated significantly better convergence, largely due to the stable clustering provided by SSL encoders. This stability facilitated successful LM training at the 0.5B scale. However, the stable clustering slightly compromised acoustic dynamics, causing only fair reconstruction quality. We further iden- tified a critical alignment flaw in Semanticodec related to AudioMAE's patch-based mechanism, where misalignment of token propagated errors throughout reconstruction. X-Codec, using Hubert-derived semantics, avoided this issue and maintained lower invalid generation probability.
19

---

Published as a conference paper at ICLR 2026
D STAGE-2: RESIDUAL MODELING
As shown in Figure 11 after Stage-1 I yields coarse semantic tokens (codebook-O), Stage-2 refines Denote the total number of codebooks by K (indexed from O to 7), Although codebook-O) is already produced by Stage-1, we train Stage-2 2 to predict all codebooks {0, 1,,,,... ..... ] jointly in a single autoregressive framework. This design ensures that the model has unified view of both the high-level structure (codebook-O) 0) and the residual details (codebooks 1-7).
the audio with additional codebooks 1, ....... 7.
Residual Tokens
Codebook
Codebook
2
Codebook
Codebook
Codebook
Stage-2 Language Model
0
Figure 11: Stage-2 Framework of YuE. S: <SOA>, S=<SOA> E=<EO>>, S.rssage.ii
Architecture Overview. x In Stage-2, we introduce additional codebooks, collectively denoted by
1:T = (z) T be the Stage- I codebook- tokens for T frames.
(1:7)
(1
(7), TT
1.T =
(0:7) For training, we treat the output space as X1. i.e. each timestep t has tuple
(0:7) = (x;
x;
Although codebook-0 -O tokens are the same as those from Stage- 1, they are included in the training target so the model learns to predict them as well, thus capturing complete frame-level dependencies across all codebooks.
Aligned Autoregressive Factorization. We maintain a strictly time-aligned factorization:
T
(0:7)
II t=1
p
(0:7)
This ensures that at each frame t, the model conditions on all previously generated tokens across all codebooks, while still maintaining frame alignment with codebook-O.
Coos-Condditioing. During training, we organize the sequence as:
(0)
(7)
12 (1)
(0)
(1)
(7)
222
all Codebook-O first
blocks of 0-7 per frame
That is, the first segment is only the codebook-O tokens, followed by repeated 8-token blocks (0.1....,7) 7) for each frame. We apply standard teacher forcing on this extended sequence and minimize
Estage2
Σ log p
(0:7) x
x (0:7))
By placing all codebook-O tokens at the beginning, the model is guaranteed to "see" the entire semantic structure before it encounters any mixed (0-7) blocks. This allows the model to plan the later residuals by attending a complete semantic outline from Stage-1.
Inference. At test time, codebook-O tokens x come from Stage-1 I and are treated as fixed (i.e., clamped). Even though the model is trained predict codebook-0 part of the joint sequence,
(0)
20

---

Published as a conference paper at ICLR 2026
during inference we replace any predicted codebook-O tokens with the Stage-1 output. Consequently, the only "free" outputs the autoregressive generation are the residual codebooks (1:7) This ensures the sequence alignment.
Implementation. Our model is 2B-parameter Transformer with an 8K-token context window, trained on consecutive 6-second single-track segments. It employs shared acoustic codebook space to model various audio types, including speech, vocals, instrumentals, and mixtures.
21

---

Published as a conference paper at ICLR 2026
E LINGUISTIC INFORMATION LOSS AFTER TOKENIZATION
We quantify Linguistic information Loss After Tokenization (LLAT) using delta Word Error Rate (AWER), defined A WER WER WERor. where WER and oER are estimated by a fine-tuned Whisper 19 model on lokemizzer-reconsttuutte.. 20 and original mixture audio, respectively. Figure [2 illustrates the relationship between AWER and music genre (hip-hop, pop, metal) using 1k sampled tracks. An upward trend is evident, with metal exhibiting the highest LLAT followed by pop and hip-hop, indicating greater modeling difficulty acoustically dense genres. Vocal-only tracks consistently achieve lower A AWR compared to mixtures, indicating lower LLAT after source separation.
AWER Across Music Genres
Momr SATR
Vical Miies
20
SWER
Vieii
Music Genre
Figure 12: AWER across different music genres for mixture / vocal-only tracks. AWER & LLAT.
A Whisper V3 checkpoint fine-tuned on an internal song dataset with manual transcription.
20We use X- X-Codec tokenizer. We use 8-codebook in this L.IAT experiment. See more discussion in
Appendix C and 2
22

---

Published as a conference paper at ICLR 2026
F SUBJECTIVE EVALUATION
F.I EVALUATION METHOOSS
In this subjective evaluation experiment, annotators were required perform pairwise comparative evaluations of music generation outputs from multiple models. Each test unit comprised two distinct musical pieces generated by different models. Following complete playback of both samples, annotators conducted binary comparative selections (options: Superiority of A, Superiority of B, or Equivalence between and B) across predefined evaluation dimensions. Mandatory preference judgments were enforced for each dimensional criterion, with explicit instructions to minimize the frequency selecting the equivalence option. The evaluation protocol incorporated a double-bline procedure with randomized presentation order of audio pairs mitigate potential ordering effects.
In the main experiments in Section 4 each model generated 42 full-length songs based on a diverse set of English prompts specifying genre, instruments, emotion, lyrics, and tempo. These prompts utilized real lyrics that were rewritten by GPT and paired with corresponding 30s chorus segments reference audio. Evaluators blindly compared pairs of music pieces produced by two different systems.
For ablation studies in Section 5 unless otherwise specified, utilize set of 15 GPT-generated English prompts (see Appendix [). Each study undergoes small-scale A/B testing, with inference performed twice per prompt, resulting total of 30 samples per setting.
Content
Label
A
O WHEN vOIE you IRINK IF mCM musical?
O Wasen wArk 7ou IhtR Iay betivr seed realiy?
Vasc
Which wark yau thik Hiche vask range?
bale and
Which wark you think better sccompaninmnt qualit?
roded
C Mall Aat
Biclado
evia
Which
money??
couuut you
Ao sabl
Mive freek demsunstratee beller
techrique?
Which
Figure 13: Subjective evaluation platform.
F.2 EVALUATION PROTOCOL
1) Overall Musicality Definition: The musical artistic value and professionalism demonstrated by the work as whole, reflecting whether it approaches the creative level of professional musicians or composers. Evaluation Criteria: Smoothness of the melody, complexity and rationality of the harmony. precision and rhythmic flow, and the artistic and creative qualities of the overall arrangement.
2) Vocal Quality (VocalQual)
Definition: The acoustic quality of vocal performance in the work.
Evaluation Criteria: Pitch, rhythmic stability, naturalness of vocal timbre (resembling human singing), fullness and warmth of timbre, degree of mechanical or distorted sound, clarity of vocals, and richness in capturing delicate emotional expressions (e.g., variations breath control, articulation precision, emotional conveyance).
3) Accompaniment Quality (AccompQual)
Definition: The acoustic quality of the instrumental accompaniment in the work. Evaluation Criteria: Realism and authenticity of instrumental timbres, dynamic variation
23

---

Published as a conference paper at ICLR 2026
and detail richness in instrumental expression (e.g., subtlety in guitar plucking pereussión dynamics).
4) Music Arrangement Complexity (MusicArr)
Definition: The layering, coherence, balance, and creativity of the musical arrangement in the work.
Evaluation Criteria: Clarity of arrangement layers, coordination and interplay between in- struments, balance of accompaniment within the overall audio track (c.g.. appropriate volume and frequency distribution), fullness of low frequencies, brightness of high frequencies, diver- sity of arrangement elements (e.g., harmony, melodic lines, rhythm patterns across multiple dimensions), creativity, and variation and emotional progression between sections.
5) Melodic Attractiveness AMelodicAttrac Definition: The memorability, accessibility, and resonance- inducing capability of the melody. Evaluation Criteria: Ease of memorization and singability, catchiness, emotional resonance, and repeated hooks or memorable elements, especially in the chorus.
Yocal-Accompaniment Compatibility NocalAAcompComp)
Definition: The consistency and compatibility between vocal melodies and instrumental accom- paniment in terms of musical style, modality, harmony, and rhythm.
Evaluation Criteria: Compatibility of vocal melodies and accompaniment in modality, har- mony, and rhythm, and absence of dissonance or conflict.
7) Song Structure Clarity (SongStruct)
Definition: The logical coherence and sectional distinctiveness of the overall song structure. Evaluation Criteria: Clarity of the song's structure (c.g., differentiation among verses, choruses, and interludes), naturalness of transitions between sections, and structural completeness.
F.3 CONDITIONAL EVALUATION DIMENSION AND DEFINITIONS
8) Lyrics Following (LyricFollow), Definition: The accuracy of Al-generated vocals in performing the lyrics specified in the prompt. Evaluation Criteria: Accuracy of lyric delivery (whether the specified lyrics are correctly performed), clarity of pronunciation (whether the lyrics intelligible), alignment of lyrics rhythm with the musical beat, and naturalness and correctness of multilingual lyric transitions and pronunciations.
9) Genre Controllability (GenCtrl) Definition: The degree to which the generated music accurately reflects the musical genre specified in the prompt.
Evaluation Criteria: Accuracy of musical genre characteristics (whether the generated music aligns with the features of the genre specified in the prompt, such as jazz, pop, classical, rock, etc.).
10) Instrument and Vocal Configuration Controllability (InstrCtrl)
Definition: The extent which the generated music adheres the instrument and vocal configuration specified in the prompt.
Evaluation Criteria: Matching of instrument and vocal configuration (whether the generated music follows the specifications in the prompt, such as piano, guitar, male or female vocals, choir, etc.).
11) Emotional Expressiveness (EmoCtrl)
Definition: The accuracy and impact of emotional expression in the generated music, as specified in the prompt.
Evaluation Criteria: Alignment of musical emotions with the emotional description in the prompt (e.g., passionate, sorrowful, cheerful).
12) Tempo and Rhythm (Tempo/Rhyytrl)
Definition: The congruence of the music's tempo (BPM) and rhythm with the requirements specified in the prompt.
Evaluation Criteria: Consistency of generated music tempo (BPM) with the tempo specified in the prompt, and adherence the required rhythmic patterns.
24

---

Published as a conference paper at ICLR 2026
YuE Udio
Suno V4
Hailuo
Tiangong
YuE Udio
Suno
Hailuo Tiangong
AccompQual
MusicAr
VocaiQual
InstrCtrt
LyricFollow
MelodicAttrac
SongStruct
EmoCtr
Tempo/RhyCtri
VocalAccompComg
Figure 14: Normalized human preference on different music aspects. Left: scores across 6 musical aspects; Right: performance on types of control.
G DETAILED COMPARISON WITH PROPRIETARY SYSTEMS ON SUBJECTIVE METRICS.
Aspects of Musicality and Acoustic Quality. To evaluate the subjective musical qualities of YuE and comparative models, we conducted a detailed A/B test on six dimensions: vocal (acoustic) quality, accompaniment (acoustic) quality, music arrangement, melodic attractiveness, matching, and song structure. We visualize the win rate with radar plot in Figure 14).). Suno V4 consistently outperforms all other models across these aspects, thus normalized the win rate by Suno to improve visual clarity. Among other models, YuE excels notably in music structure and music arrangement, highlighting its capability for coherent long-form composition capability. However, YuE shows clear deficiencies in vocal and accompaniment acoustic quality, likely due to limitations of its current audio tokenization method. While YuE achieves decent musicality and convergence. the semantic-fusee tokenizer requires improvements in acoustic detail via an enhanced decoder or super-resolution backend.
Controllability. Similarly, evaluated the controllability of YuE and comparative models through A/B testing on five dimensions: genre control, instrument/vocal control, emotion control, tem- po/rhythm control, and lyrics following. Given limitations of existing classifiers and transcription systems, user preference win rate was our primary evaluation metric, with results presented in Fig 14(R). Suno consistently outperforms all models across controllability metrics. Among other models, YuE performs strongest in genre adherence, instrument/vone highlighting effectiveness generating stylistically coherent music aligned with textual prompts. YuE demonstrates moderate performance in emotion and tempo control, indicating the need for improved lyric alignment and tempo tagging systems due to considerable noise observed in the pseudo label on the training corpus provided by Qwen2Audio. Overall, these results affirm YuE's robust controllability capabilities.
consistency, and emotion,
H CORRELATION BETWEEN AUTOMATIC METRICS AND HUMAN EVALUATION
Correlation with Musicality & Average Preference When considering musicality and average human preference (Table 7. the Vocal Range metric stands out, correlating most strongly (above 0.85) with both subjective ratings. This highlights the crucial role of vocal expressiveness and melodic
25

---

Published as a conference paper at ICLR 2026
Table 7: Pearson correlation between subjective metries (Musicality, Average) and automatic metrics. Vocal Range strongly impacts Musicality and Average ratings.
KL
FAD
CE
CU
PC
PQ
CLAP
CLaMP 3
VocalRange
Musicality
-0.232
-0.249
0.368
0.320
-0.268
0.112
-0.072
0.333 0.264
0.857
Average
-0.199
-0.351
0.357
0.303
-0.128
0.054
0.086
0.858
21 diversity in listeners' overall impressions of generated music. We find vocal range to be a practical proxy for musicality and recommend its adoption.
Table 8: Pearson correlation of alignment metrics vs, human preference on controllability.
LyricFollow
GenCtrl
InstrCtrl
EmoCtrl
Tempo/RhyCtrl
CLAPt CLaMP 31
-0.25 0.42
0.01
-0.07
0.14
0.09
0.37
0.44
0.33
0.36
Alignment Metrics. The correlation results in Table demonstratee that CLaMP 3 scores consis- tently correlate better with human evaluations of controllability compared CLAP scores. This is particularly evident in tasks such as I LyricFollow (0.42 vs. -0.25) and InstrCtrl (0.44 vs. -0.07). Interestingly, the genre-following capability measured by the CLaMP 3 backend (Wu et al.; 2025) appears to be closely related to lyric- syric-following performance, even though lyrics are not explicitly included in the computation of the CLaMP 3 score. This indicates a correlation between genre controllability and lyric adherence in music generation models. Conversely, the weaker correlations observed with CLAP suggest limitations in its capacity to capture nuanced perceptual aspects, likely due to insufficient exposure to singing and music-specific content during pre-training.
Table 9: Pearson correlation of KI. and FAD acoustic quality preference metries.
AccompQual
VocalQual
KL FAD
0.14
0.23
-0.15
-0.11
Distribution Matching Metrics. We employed the more advanced PaSST (Koutini et al. 2021
backbone instead of the conventional VGGish (Hershey et al.) 2017) to evaluate distribution matching metrics. Despite its sophistication, the AudioSet pre-trained backbone may inherently suffer from out-of-distribution (OOD) issues when dealing with generative music, particularly with singing or vocal elements. Additionally, sample size bias may contribute significantly, limited availability of extensive audio samples from closed-source generative systems hinders accurate distribution estimations.
As shown in Table both KL and FAD exhibit weak correlations with accompaniment (acoustic) quality (Accompuual) and vocal (acoustic) quality (VocalQual), suggesting that distribution-level metrics may not fully capture subtle subjective perceptions of acoustic fidelity in our case. However, as indicated in Table 7 these same metrics correlate more strongly with musicality and overall human preference
22 This implies that while distribution matching may not always reflect finer acoustic details, they sometimes reflect qualities relevant to perceived musicality and listener satisfaction.
One possible explanation relates to AR music generation behavior. Such models often favor high- probabiiyyy tokens, biasing melodies toward conservative choices like tonic, chord tones, or previously generated notes. Poor optimization (c.g., overfitting) or overly conservative sampling exacerbates this issue, reducing melodic diversity.
Both KI. and FAD negatively correlated, since lower values indicate better alignment.
26

---

Published as a conference paper at ICLR 2026
Content-Based Metrics. In Table 10 CE exhibits the strongest correlations, particularly with subjective acoustic quality measures such as VocalQual (0.66) and AccompQual (0.56). This indicates that CE might be especially sensitive acoustic characteristics perceived by listeners. By contrast, correlations with musicality-relate aspects-such as SongStruct (0.33), VAComp (0.35), MelAttrac (0.30), and MusicArr (0.31)-ah) are relatively lower, suggesting lesser sensitivity of CE to detailed musical attributes. Meanwhile, both PC and PQ show notably weaker or inconsistent correlations these subjective metrics, implying limitations in their ability to capture musicality related perceptual elements.
Table 10: Pearson correlation content-based metries vs. related preference metrics.
AccompQual
VocalQual
SongStruct
VAComp
MelAttrac
MusicArr
CE
0.56
0.66
0.33
0.35
0.30
0.31
CU
0.50
0.61
0.27
0.29
0.25
0.26
PC
-0.09
0.00
-0.24
-0.20
0.00
-0.16
PQ
0.27
0.36
0.05
0.06
-0.03
0.02
I
15 ENGLISH PROMPTS FROM GPT
ID:
[Genre] Rap
[verse]
Woke up the morning, sun shining bright
Chasing all my dreams, gotta get my mind right
City lights are fading, but my vision's 's clear
Got my team beside me, streets, beats inside my head Every step take, closer to the bread People passing by, they don't 't understand
for fear
Walking through
Building up my future with my two hands
[chorus]
This is my life, and I'm aiming for the top Never gonna quit, no, I' I'm never gonna stop Through the highs and lows, I' m'a keep real Living out my dreams with this mie and deal
[verse]
Late nights grinding, writing down these rhymes Clock is ticking fast, can't afford waste time
brush it off Turn the negativity into something strong Mama working hard, wanna make her proud Echoes of her prayers cutting through the crowd Friends turned strangers, but it's all good
Haters gonna hate,
Focused on my path like always knew would
[chorus]
This is my journey, and I'm running this race Heart full of fire, you can see my face Obstacles ahead, but got no fear Victory is close, yeah, it's almost here
[bridge]
They said I couldn't do it, said I'd never rise
But now I'm soaring high, reaching for the skies
Lessons that learned made me who I am
27

---

Published as a conference paper at ICLR 2026
Standing tall now, don't give damn
[verse]
Bchoes in the alley, music's 's getting loud Feeling the adrenaline pumping through the crowd Spollights me, it's my time shine Living in the moment, everything's aligned Looking back now at the roads I've crossed Every single battle, every line I've tossed Made stronger, wiser, ready for what's next Writing my own story, turning pages of the
[chorus]
This is my song. and I'm singing it proud Voices united, hear shout out loud From the underground straight into stars Carving out my name, lcaving all these sears
[outro]
Yeah, this is for the dreamers, the who never quit
Keep your head up high, and don't you ever submit
Life is what you make it, so make it something great Step into your purpose, and seize your fate
28

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Rock
[verse]
shadows in the night The city's heartbeat echoes under lights Hands deep in pockets, wandering alone Footsteps tracing paths to the unknown Suddenly he pauses. looks up the sky Eyes reffect questions passing by Whispers to the wind, words without sound Scarching for the answers yet unfound
Standing on the
[chorus]
Lost within chaos, seeking out sign In a world of color, drawing blurred lines Moving forward, looking back, unsure of the way Trying to find place where he stay
[verse]
He crosses empty streets, under neon glow Faces in the crowd, stories left untold Raises up his arms, reaching for the truth Grasping at fragments his youth Billboards and the banners flutter in the breeze The rhythm of the city brings him to his knees Hcartbeat heavy, nowhere left to hide
Feeling like he's lost amidst the tide
[chorus]
Lost within the chaos, seeking out sign In world of color, drawing blurred lines Moving forward, looking back, unsure of the way Trying to find a place where he can stay
[bridge]
Doesn't t want leave, doesn't want fight
Caught between the darkness and the light
No need for reason, nothing to prove Just soul in transit, with nothing lose
[outro]
Doesn't 't want to leave, doesn't 't want to fight
Chasing after shadows in the night
He doesn't need the truth, doesn't t need name
Just looking for spark fan the flame
29

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Pop
[verse]
Staring the sunset, colors paint the sky Thoughts you keep swirling, can't deny
I know let you down, I made mistakes But I'm here to mend the heart didn't tt break
[chorus]
Every road you take, I'll be one step hehind Every dream you chase, I'm reaching for the light You can't t fight this feeling now I won't back down the whisper
wind, the shadow by your side The warmth you feel within when you can't hide You know you can't it deny now
I won't back down
[verse]
They might say I'm faolish, chasing after you
But they don't It feel this love the way we do
My heart heats only for you, can't you see? won't 't let you slip away from me
[chorus]
Every road you take, I'll be one step behind
Every dream you chase, I'm reaching for the light You can't 't fight this feeling now I won't back down I'm the whisper in the wind, the shadow by your side The warmth you feel within when you can't hide You know you can't 't deny now
I won't back down
[bridge]
No, won't back down, won't around
Until you't 're back where you belong
I'll cross the oceans wide, stand by your side Together we are strong
[outro]
Every road you take, I'll be one step behind
Every dream you chase, love's 's the tic that binds
You can't t fight this feeling now
I won't back down
30

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Jazz
[verse]
In the quiet of the evening, shadows fall Whispers of the night wind echo through hall Lost within the silence, hear your gentle voice
Guiding me back homeward, making my heart rejoice
[chorus]
Don't this moment fade, hold me close tonight
With you here beside me, everything's 's alright
Can't 't imaginc life alone, don't want to let you go
Stay with me forever, let our love just flow
[verse]
Moonlight paints picture upon your lovely face Every glance between us fills the empty space Time stands still around us when you'te 're my arms Nothing else can matter, safe from any harm
[chorus]
Don't 't let this moment fade, hold close tonight
With you here beside me, everything 's alright Can't imagine life alone, don't want to let you go
Stay with me forever, let our love just flow
[bridge]
Every touch ignites a fire, burning deep within
Every smile you give to me makes my head spin
Promise you'l Il stay awhile, don't ever say goodbye the sky
Together we'll lll chase every
[chorus]
Don't 't let this moment fade, hold me close tonight
With you here beside me, everything's alright let you go Stay with me forever, our love just flow
Can't 't imagine life alone, don't 't
[outro]
Stay with me forever, let love just flow
31

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Blues
[verse]
Late last night, the rain was pouring down Lonely footsteps echoed through the town Thinking 'out the love that slipped away Wondering how I let you go that day
[chorus]
Oh, my angel, where have you flown Left me here to face this world alone I'm just fool, a fool in love with you Can't I deny this heartache's 's true
[verse]
Strectlights flicker, shadows on the wall Memories of you, recall
Your laughter like song inside my head
Without you here, my soul feels dead
[chorus]
Oh, my angel, won't you this love, still burn I'm just a fool, a fool in love with you Hoping someday you'l 'll feel it too
[bridge]
fell for you, and always knew That my world revolves around you
I hope and I pray, both night and day That you'll 'll come back and choose to stay
[chorus]
Oh. my angel, where have you flown
Left me here to face this world alone I' I'm fool, fool in love with you Waiting here, what else can do
[outro]
I'm just fool, fool in love with you
32

---

Published as a conference paper at ICLR 2026
ID:
[Genre] RnB_Soul
[verse]
Why don't just find place hide Leave our worries and doubts behind
When nothing in this world is as seems Together we can live inside our dreams There's need to be afraid tonight the love ve made, we'll find light When we're re living world of our own It's you and me, we never feel alone
[chorus]
They say it's hard man to let it show But with you, I'm ready to let all go Whatever we try, we' re gonna get there You take control, baby, don't care gotta keep on pushing when times get tough
We keep on making better love
[verse]
Don't believe the things that others say We've tried all found our way They should take a look at you and me Learning how to love and set it free For every hcartache,
take You teach me yours and I'll show you mine About the way that love is meant to be Together we' rewrite our history
time
[chorus]
They say hard man let it show But with you, I'm ready to let all go Whatever we try, we'r re gonna get there You take control, baby, don't care I gotta keep pushing when times get tough keep on making better love
[bridge]
Gotta take control and swallow my pride
Every man has feelings deep inside You gotta find yourself before you can Be ready to love and understand Baby, know what you'le 're thinking of
We keep making better love
[outro]
I believe the love we'r re making's 's gonna last forevermore
Loving you feels right, like never before
We'll be getting down tonight until the morning light
We keep on making better love Better love, we'll be making
Better love, no more faking They say it's hard for But with you here, I'm ready to let go Whatever we try, we're gonna get there You take control, baby, don't care I gotta keep pushing when times get tough We keep making better love Better love (till fade out)
let it show
33

---

Published as a conference paper at ICLR 2026
ID:
[Genre] SeeiinttChinese.SSyy
[verse]
Beneath the moonlit sky vast
lone wanderer recalls past
Whispers of the bamboo leaves
Echo tales the wind retrieves
[chorus]
Oh, the rivers flow, mountains high Journeying souls beneath the endless sky
Threads of fate entwine our way
Guiding us through night and day
[verse]
Lanterns glow with softest light Painting shadows in the night Silken robes and ancient songs Memories where hearts belong
[chorus]
Oh. the rivers flow, mountains high Journeying souls beneath the endless sky Threads fate entwine our way
Guiding us through night and day
[bridge]
reflect in tranquil ponds
Dreams unfold of times beyond
Lotus blooms and cranes in flight
Secrets held within the night
[outro]
As the sunrise paints the east Bringing hope and inner peace Footprints fade upon the shore
But the spirit journeys evermore
34

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Folk
[verse]
Underneath the open sky clear, We gather 'round with voices near. Through trials faced and stories told, Our spirits rise, our hearts unfold.
[chorus]
So lift the lanterns to the sky,
Together we will soar and fly. Though shadows loom and doubts appear,
We'll keep the flame forever here.
[verse]
Remember all the paths we're ve crossed, The battles won, the moments lost. banner of hope we hold up high,
symbol shining in our eyes.
[chorus]
So lift the lanterns the sky,
Together we will soar and fly.
Though shadows loom and doubts appear,
We'll Il keep the flame forever here.
[bridge]
With hands united, we stand tall, Pledged to rise we should fall.
Through darkest nights and stormy seas, Our song will carry the breeze.
[chorus]
So lift the lanterns to the sky. Together we will soar and fly.
Though shadows loom and doubts appear,
We'll keep the flame forever here.
[outro]
We'll keep the flame forever here.
35

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Dance
[verse]
Underneath starlit sky,
come alive, you and
City lights are shining bright, Dancing through the endless night.
[chorus]
Who are we? Let's break away, Feel the beat and let it play. Lost in music, hearts align, this moment, we define.
[verse]
Shadows fade beneath the glow, Rhythms guide us where to go.
Voices whisper the crowd,
Tur it up. we' 'll sing aloud.
[chorus]
Who we? Let's break away, Feel the beat and play. Lost music, hearts align.
In this moment, we define.
[bridge]
Let the melody surround, Lift us off the solid ground.
Every step and every move.
In this dance find groove.
[chorus]
Who are we? Let's 's break away. Feel the beat and let play. Lost in music, hearts align, this moment, we define.
[outro]
Keep dancing, feel the heat,
Moving the pounding beat.
Who we are is here and now, Take my hand, we'll Il show them how.
36

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Country
[verse]
Da-dum, da-da-da--ladda--ddda
Da-dum, da-dum
Walking down this lonesome road Thinking 'bout the love untold Why haven't 't told you I've whispered the midnight stars Just how wonderful you are Why haven't told you
[chorus]
Friends keep asking I'm fine I just smile and say you'le re mine
Might as well confess Can't keep this inside Maybe you feel the same way too Oh darling. if you do
Why haven't t you told me
Da-dum, da-
[verse]
I've sung it to the morning sun That with you, my life's begun Why haven't t told you
My heart's an open book today Waiting for the words to say Why haven't told you
[chorus]
Friends keep asking what's the news I just grin and think of you
Time to take chance Let my feelings show Mayhe you feel the way Oh darling, you
Why haven't you told me
[bridge]
Da-dum, da-da-da-da-da-dar
Da-dum, da-dum
No more holding back these words
Let them fly just like the birds
[chorus]
Now I'm standing here tonight
Hoping that got it right
Might well confess
Can't keep this inside Maybe you feel the same way too Oh darling, if you do
Let's not waste another day
Why haven't told
[outro]
Da-dum, da-da- da da-da- da---addaaaa da-da
Da-dum, da-dum
Now we've ve finally told us
Our new life's begun
37

---

Published as a conference paper at ICLR 2026
ID: 11
[Genre] Rap
[verse]
Woke up the morning,
shining bright
Chasing all my dreams, gotta get my mind right
City lights are fading, but my vision's clear
Got team beside me, no room for fear Walking through the streets, beats inside my head Every step I take, closer to the bread People passing by, they don't understand
Building up my future with my own two hands
[chorus]
This my life, and aiming for the top Never gonna quit, no, I'm never gonna stop Through the highs and lows, I' 'mmma keep real Living my dreams with this mic and deal
[verse]
Late nights grinding, writing down these rhymes Clock is ticking fast, can't 't afford to waste time Haters gonna hate, but brush it off Turn the negativity into something strong Mama working hard, wanna make her proud Echoes of her prayers cutting through the crowd Friends turned strangers, but it's all good
Focused on my path like always knew would
[chorus]
This is my journey, and I' I'm running this race it my face
Heart full of fire, you
Obstacles ahead, but got fear Victory is close, yeah, it's almost here
[bridge]
They said I couldn't 't do it, said I'd rise But now I'm soaring high, reaching for skies Lessons that learned made me who I am Standing tall now, don't give damn
[verse]
Echocs in the alley, music's 's getting loud Feeling the adrenaline pumping through the crowd Spotlights me, it's my time to shine Living in the moment, everything's aligned Looking hack now at roads I've crossed Every single battle, every line I've tossed Made me stronger, wiser. ready for what's next Writing my own story, turning pages of the text
[chorus]
This is my song, and I'm singing it proud Voices united, hear shout out loud From the underground straight into the stars Carving out my name, leaving all these scars
[outro]
Yeah, this is for the dreamers, the who never quit
Keep your head up high, and don't 't you ever submit
Life is what you make it, so make it something great
Step into your purpose, go and seize your fate
38

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Rock
[verse]
shadows in the night The city's heartbeat echoes under lights Hands deep in pockets, wandering alone Footsteps tracing paths to the unknown Suddenly he pauses. looks up the sky Eyes reffect questions passing by Whispers to the wind, words without sound Scarching for the answers yet unfound
Standing on the
[chorus]
Lost within chaos, seeking out sign In a world of color, drawing blurred lines Moving forward, looking back, unsure of the way Trying to find place where he stay
[verse]
He crosses empty streets, under neon glow Faces in the crowd, stories left untold Raises up his arms, reaching for the truth Grasping at fragments his youth Billboards and the banners flutter in the breeze The rhythm of the city brings him to his knees Hcartbeat heavy, nowhere left to hide
Feeling like he's lost amidst the tide
[chorus]
Lost within the chaos, seeking out sign In world of color, drawing blurred lines Moving forward, looking back, unsure of the way Trying to find a place where he can stay
[bridge]
Doesn't t want leave, doesn't want fight
Caught between the darkness and the light
No need for reason, nothing to prove Just soul in transit, with nothing lose
[outro]
Doesn't 't want to leave, doesn't 't want to fight
Chasing after shadows in the night
He doesn't need the truth, doesn't t need name
Just looking for spark fan the flame
39

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Pop
[verse]
Staring the sunset, colors paint the sky Thoughts you keep swirling, can't deny
I know let you down, I made mistakes But I'm here to mend the heart didn't tt break
[chorus]
Every road you take, I'll be one step hehind Every dream you chase, I'm reaching for the light You can't t fight this feeling now I won't back down the whisper
wind, the shadow by your side The warmth you feel within when you can't hide You know you can't it deny now
I won't back down
[verse]
They might say I'm faolish, chasing after you
But they don't It feel this love the way we do
My heart heats only for you, can't you see? won't 't let you slip away from me
[chorus]
Every road you take, I'll be one step behind
Every dream you chase, I'm reaching for the light You can't tt fight this feeling now I won't back down I'm the whisper in the wind, the shadow by your side The warmth you feel within when you can't hide You know you can't 't deny now
I won't back down
[bridge]
No, won't back down, won't around
Until you't 're back where you belong
I'll cross the oceans wide, stand by your side Together we are strong
[outro]
Every road you take, I'll be one step behind
Every dream you chase, love's 's the tic that binds
You can't t fight this feeling now
I won't back down
40

---

Published as a conference paper at ICLR 2026
ID:
[Genre] Jazz
[verse]
In the quiet of the evening, shadows fall Whispers of the night wind echo through hall Lost within the silence, hear your gentle voice
Guiding me back homeward, making my heart rejoice
[chorus]
Don't this moment fade, hold me close tonight
With you here beside me, everything's 's alright
Can't 't imaginc life alone, don't want to let you go
Stay with me forever, let our love just flow
[verse]
Moonlight paints picture upon your lovely face Every glance between us fills the empty space Time stands still around us when you'te 'rere my arms Nothing else can matter, safe from any harm
[chorus]
Don't 't let this moment fade, hold close tonight
With you here beside me, everything 's alright Can't imagine life alone, don't want to let you go
Stay with me forever, let our love just flow
[bridge]
Every touch ignites a fire, burning deep within
Every smile you give to me makes my head spin
Promise you'l Il stay awhile, don't ever say goodbye the sky
Together we'll lll chase every
[chorus]
Don't 't let this moment fade, hold me close tonight
With you here beside me, everything's alright let you go Stay with me forever, our love just flow
Can't 't imagine life alone, don't 't
[outro]
Stay with me forever, let love just flow
41

---

Published as a conference paper at ICLR 2026
ID: 15
[Genre] Blues
[verse]
Late last night, the rain was pouring down Lonely footsteps echoed through the town Thinking 'out the love that slipped away Wondering how I let you go that day
[chorus]
Oh, my angel, where have you flown Left me here to face this world alone I'm just fool, a fool in love with you Can't I deny this heartache's 's true
[verse]
Strectlights flicker, shadows on the wall Memories of you, recall
Your laughter like song inside my head
Without you here, my soul feels dead
[chorus]
Oh, my angel, won't you this love, still burn I'm just a fool, a fool in love with you Hoping someday you'l 'll feel it too
[bridge]
fell for you, and always knew That my world revolves around you
I hope and I pray, both night and day That you'll 'll come back and choose to stay
[chorus]
Oh. my angel, where have you flown
Left me here to face this world alone I' I'm fool, fool in love with you Waiting here, what else can do
[outro]
I'm just fool, fool in love with you
42


Reviews:
[{"review_id": "9d9cjpEu0J", "reviewer": "Reviewer_aT19", "summary": "This paper presents YuE, a lyrics-to-song generation model that aims to produce long-form music generation with vocals and accompaniment. The authors propose a two-stage autoregressive architecture operating on discrete music tokens (X-Codec). The first stage predicts the first codebook of the music tokens by givens the semantic conditions (lyric, and others). and the second stage makes up the token predictions of the higher codebooks. To support long-form generation, the authors introduce Structural Progressive Conditioning (SPC), which uses section-level structure tokens to progressively condition the LM. Experimental results demonstrates that YuE performs competitively with commercial models (Suno, Udio, Hailuo).", "questions": "1. Why is SongBloom not included in your comparisons? Can you add some comparisons to it (and ACE-Step) and also give the evidence on the difference between your potential results to the results reported in their papers?\n\n2. Can you provide detailed documentation of the 650k hours of CC-licensed music? Which platforms were used and under what specific license? How many tracks / hours from each source? Are licenses compatible with redistribution + model training? Do you own the copyright of using these data? If not, since you release the model, how can external users ensure they are free from copyright exposure? How can the community safely adopt YuE?\n\n3. Please provide ablation studies including: Dual-NTP vs. single (combine voice and accompaniment) token prediction; SPC vs. no-SPC; and Data-scaling sensitivity (e.g., 10k/100k hours). For the data-scaling, it is encouraged to use the open-available data for training, even the results are not good (e.g., FMA, Jamendo, MusDB, MoisesDB, MedleyDB). Other data contains the lyric annotation might better to be used. \n\n4. For reproducibility, it is encouraged to have example data from your training set, and will you release the training script with data loader and processing, or just a simple inference code and model checkpoint?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper has two strengths.\n\n1.The usage of long-form structure conditioning (SPC) is novel. Using section metadata to segment data and guide generation is a practical strategy for improving macro-level structure, and the staged conditioning improves continuity.\n\n2.The experimental design is reasonable and comprehensive. The paper includes both objective metrics and human preference studies among YuE and four baselines. The results (and the demo website) show multi-minute outputs with acceptable alignment and structure.", "weaknesses": "The paper has several critical weaknesses that hinders its acceptance.\n\n1.The overall novelty of this paper is limited, especially compared to the same period and already published work (SongBloom [1]) in NeurIPS 2025. Specifically, the two-stage autoregressive LM with Dual-NTP and SPC is incremental relative to SongBloom. And SongBloom even proposed more modern formulation: \n\n1 (a). SongBloom applies a unified autoregressive-diffusion architecture that interleaves semantic sketching and acoustic refinement, enabling significantly stronger acoustic quality with comparable modeling complexity. \n\n1 (b). SongBloom also covers the voice and accompaniment tokens by leveraging the Demucs separation model to extract the music data.\n\n1 (c). SongBloom covers essentially all YuE capabilities and more, including lyric-to-music generation, structured multi-section music, reference audio prompting, diffusion decoder for refinement, disentanglement of semantic and acoustic tokens, and also open-source release.\n\n1 (d). Despite having very similar high-level goals and tokenization methods, YuE does not compare to SongBloom, although SongBloom explicitly compares against YuE and reports better performance.\n\nCompared to SongBloom, YuE’s design is behind state-of-the-art, which lacks diffusion-based refinement, Interleaving of semantic + acoustic token, and robust modeling of reference-prompt conditions. Therefore, YuE does not demonstrate novelty or superiority over SongBloom. At present, it is difficult to justify acceptance when a clearly more advanced method has already been published.\n\n2.The experimental design is comprehensive but the baseline design is problematic. Apparently, YuE does not compare to the most relevant works, including SongBloom and ACE-Step [2]. Although YuE includes industry comparisons (Suno, Udio, Tiangong), the lack of comparison to academic/open models with similar architectures weakens the empirical claims. Also there is no ablation studies to isolate the contributions of Dual-NTP (decoupled vs non-decoupled), SPC, and Data-scale effects. And such ablation studies are very imperative to fully demonstrate YuE's contribution and its improvement to prior works. From the experimental results, YuE does not demonstrate superiority. It does not outperform Suno V4, and remain slightly behind Udio and Tiangong. It is acceptable that the model remains competitive but not superior to industrial models. But to fully demonstrate the proposed method, more ablation studies and comparisons to academical or open-source models should be considered. At the same time, SongBloom reports stronger performance over YuE. Thereofore, YuE does not establish SOTA status for either research or production settings.\n\n3.The data sourcing transparency and licensing concerns remain vague in the paper. The paper claims to train on around 650k hours of creative common (CC) music. However, there are several critical concerns. The actual data sources are not specified. The platforms, artists, catalogs remain unknown, and the licenses are not enumerated (CC-BY and CC-BY-NC are completely different). It is unclear whether scraping is permitted or whether data came from commercial music platforms (plus that, whether the authors of YuE own the copyrights of these data for AI training). And there is no dataset preview or download interface provided. So reproducibility, legality, and downstream use constraints remain ambiguous. Given active litigation around music AI, such as Suno and Udio to three major labels (Sony, UMG, and Warner), this raises concerns about copyright and reproducibility. Till now, only Udio and UMG come to an agreement of data for music AI training. Open-source models must satisfy a higher legal/ethical bar, and YuE’s data description is too vague to be evaluated.\n\n4.The claim and the vibe of novelty overstates the contribution. The paper claims that “As of its release on Jan. 28, 2025, YuE familiy is the first publicly available, open-source lyrics-to-song model capable of full-song generation with quality on par with commercial systems\". As a reviewer, this makes an excuse to prevent it from comparing to latest models after that. This is negative and questionable, especially given SongBloom’s public release and NeurIPS-publication timeline. The lack of acknowledgment or direct comparison reduces credibility.\n\n\n[1] SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement. NeurIPS 2025.\n\n[2] ACE-Step: A Step Towards Music Generation Foundation Model", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z0bIXDLr4Y", "reviewer": "Reviewer_ytHG", "summary": "The paper introduces YuE, a family of open-source foundation models designed for long-form music generation, and more specifically, the lyrics-to-song problem. The model is trained on trillions of tokens and can generate coherent, lyrically-aligned songs of up to five minutes in length. YuE achieves its performance through several key technical contributions:\n\n- Structural Progressive Conditioning (SPC): This approach uses the inherent musical structure (like verses and choruses) to enable long-context lyrical alignment and overall song structure control, which is essential for full-song generation.\n- Redesigned In-Context Learning (ICL): A novel framework for music that allows for advanced use cases such as style transfer, voice cloning, and bidirectional content creation.\n- Track-Decoupled Next-Token Prediction (Dual-NTP): A variation of existing multi-track/multi-stem music generation methods here adapted for jointly generating the vocal and accompaniment tracks. The method is reported to improve lyrical intelligibility and to overcome problems related to dense signal mixtures.\n\nThrough extensive human evaluation, the model is shown to be competitive with, and in some aspects, surpasses widely used proprietary music-generation systems (like Tiangong and Udio) in musicality and vocal agility.", "questions": "Please see under weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### Strength\n\n- innovative SPC training strategy taking into account musical structure\n- successful evaluation demonstrating competitive performance with state-of-the-art music generation models \n- open source availability", "weaknesses": "### References are outdated, discussion of relevant existing approaches is incomplete. \n\n- paper states in the abstract to refer to the state when the model was initially published in January 2025.\n- The list of references appears to be limited when it comes to covering more recent approaches. However, relevant approaches have appeared early in 2025. Here we consider prioublciation in 2026. Therefore appacohes from early 205 should be included into the discussion \n\n- NTP decoupled next token prediction. Rather similar approaches have been proposed here:\n\nLiu, 2025. \"SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation\", https://arxiv.org/pdf/2502.13128\nRouard, 2025. \"MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling\" https://arxiv.org/pdf/2501.01757\n\nI think these papers should be discussed under related works, and they should be compared to the proposed NTP strategy. \n\n- concerning SPC a related approach has been published in\n\nLam, 2025. \"MusiCoT: Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generation\", https://arxiv.org/pdf/2503.19611\n\nThis alternative approach for including musical structure into music generation should at least be mentioned. I note that the paper features a comparison with YuE.\n\n- ICL\n\nThe paper makes the point of comparing with speech/TTS ICL methods. Given today, there exist a few approaches supporting lyricx conditioning for music generation, it appears a bit outdated to compare with these methods from a different problem domain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CCdA8fJSRO", "reviewer": "Reviewer_wdiX", "summary": "This paper presents YuE for lyrics to song generation. It aims to generate full songs (up to five minutes) with vocals, accompaniment, and makes sure that the lyrics would align with the melody/companion.\n\nThe model is based on a two-stage autoregressive framework using discrete audio codec. The first stage models lyrics to first layer audio tokens, and then the second stages models first layer tokens to all residual tokens. \n\nThe paper introduces three key techniques:\n\nDual-NTP: Each timestep in the first stage LM generates one vocal token and one companion token, instead of directly modeling the mixture.\n\nSPC: A new way to organize training data. Songs are segmented into sections (intro, verse, chorus, etc.), and their lyrics and audio tokens are interleaved section by section. This helps the model maintain long-range lyrical alignment.\n\nMusic ICL (In-Context Learning): A modified in-context learning setup where a 30-second reference audio is prepended to SPC data.\nFor evaluation, YuE performs comparably to Tiangong and Udio, better than Hailuo, but below Suno V4. Objective metrics (KL divergence, FAD, alignment scores) also show strong performance.\n\nFor evaluation, YuE performs comparably to Tiangong and Udio, outperforming Hailuo but falling short of Suno V4. Objective metrics (KL divergence, FAD, alignment scores) also show strong performance.\n\nOverall, YuE is positioned as the first open-source full-song lyrics-to-song model with quality approaching commercial systems.", "questions": "1.\tHow are the tracks separated from the mixture? Does the separation process introduce perceivable artifacts that could impact performance? What separation model is being used?\n\n2.\tWhat is the dataset used in detail? Is there any preprocessing to clean the dataset?\n\n3.\tFor the dual-track tokens, the voice should have much lower entropy than the accompaniment. Have you considered using a low-bitrate codec for voice?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tThis work presents the first open-sourced song generation model that achieves performance close to commercial systems.\n2.\tImplementing and training a full-length audio LM of this scale is a major technical achievement.\n3.\tDual-NTP and SPC integrate domain-specific musical structure into the generative process, representing a meaningful methodological contribution.\n4.\tDespite limited access to proprietary data, YuE achieves results comparable to those of closed-source commercial models", "weaknesses": "1.\tThe descriptions of data preprocessing are somewhat brief. It would help to clarify how vocal–accompaniment tracks were separated and how Creative Commons data were curated.\n\n2.\tThe “all-in-one method” used in the paper could be elaborated.\n\n3.\tThe paper could provide more analysis of sample diversity and potential copyright overlap (e.g., unintentional melodic copying). Clarifying whether such issues were observed would strengthen credibility.\n\n4.\tPrior research on multi-track or separated-source generation [1–3] should be acknowledged to better contextualize the contribution.\n[1] Mariani, Giorgio, et al. \"Multi-Source Diffusion Models for Simultaneous Music Generation and Separation.\" The Twelfth International Conference on Learning Representations.\n[2] Xu, Zhongweiyang, et al. \"Multi-Source Music Generation with Latent Diffusion.\" Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation.\n[3] Karchkhadze, Tornike, et al. \"Multi-track musicldm: Towards versatile music generation with latent diffusion model.\" International Conference on ArtsIT, Interactivity and Game Creation. Cham: Springer Nature Switzerland, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EWDf25o2Fg", "reviewer": "Reviewer_szRj", "summary": "This paper proposes a generative model for lyrics to full song generation.\nThe authors observe that generating mixed tracks (voice + accompaniment) directly leads to a loss of intelligibility, even more pronounced for some musical genres. The proposed approach to overcome this issue is to rely on source separation to decompose tracks into vocals + accompaniment and to encode each component separately.\nThe vocals and accompaniment tokens are then interleaved so that the authors can rely on a standard autoregressive (based on Llama 2) next token prediction. In practice, only the first codebook of the RVQ representations is modelled in this stage, and a second stage consists in predicting the remaining RVQ codes.\n\nThere's an accompanying website as well as an open source + weights release.\nEvaluations are thorough and go beyond simple audio-quality measures, with a real emphasis on vocals.", "questions": "- Is the dataset available?\n- For the 30s context given for in-context learning, do you make sure to consider a part that is different (like not providing Verse 2 to generate Verse 1) as these are expected to be extremely similar for the instrumental parts?\n- How is it possible to do condition on a 30s context and ask to generate the full song?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Modelling choices are coherent and well motivated: e.g. generating first interleaved vocals-accompaniment RVQ tokens, splitting full-song generation into multiple segments.\n- Open weights + open source is a plus, as song genAI models with vocals are rare. Authors also discuss the ethical issue and explain their scraping process and the questions this raises.\n- Extensive experiments\n- Many details and discussions about dataset, observations, evaluation.", "weaknesses": "Some parts could be better explained like MUSIC IN-CONTEXT LEARNING, this introduces many notations that are not useful.\nStructural progressive conditioning is also evoked too quickly.\nMaybe being clearer in the main text that instead of lyrics to song, the problem is in fact casted as a \"lyrics with structure\" to song. This is clear in appendix I.\n- The way ICL is used (i.e. by conditioning on an existing song chorus) highly conditions the type of possible applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)