ngquocvinh/YuE2-3B-GGUF

YuE2-3B GGUF

Community GGUF quantizations of m-a-p/YuE2-3B.

☕ If this GGUF made your day easier, a coffee would make mine.
Send a coffee ☕
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.

Upstream benchmark context

Frontier song quality and text alignment on 192 WildSongBench prompts. YuE2 uses symbolic planning; Bo8 means best-of-8.

This is the upstream YuE2 figure, reproduced for context and linked to the original model card. It is not a fidelity measurement of these GGUF files.

About YuE2

YuE2 is a lyrics-and-style-conditioned music generation model. The upstream model combines symbolic planning with AR/NAR music-token generation and a VAE decoder; its model card describes 48 kHz stereo music output and English and Chinese support. See the official YuE2-3B model card and the YuE2 VAE model card.

This package uses the native audio.cpp yue2 loader. The main model and VAE are separate GGUF components, and the included sidecars must remain beside them.

The release is split into two clearly labeled paths: native audio.cpp GGUF inference for the files below, and the upstream Python BF16 reference under reference/official-python/. The Python path is included for users who want the official staged API and source; it is not a GGUF runtime.

Files

File	Size	Description
yue2-3b-bf16.gguf	6.76 GiB	Main YuE2-3B BF16 reference.
yue2-3b-q8_0.gguf	3.97 GiB	Highest-precision practical profile.
yue2-3b-q8_0_f16.gguf	4.69 GiB	Audio.cpp-native high-precision mixed profile: Q8_0 base with selected AR/NAR value and down projections in F16.
yue2-3b-q6_k.gguf	3.25 GiB	High-quality K-family profile.
yue2-3b-q6_k_m.gguf	3.44 GiB	Audio.cpp-native mixed profile: Q6_K base with selected AR/NAR value and down projections in Q8_0.
yue2-3b-q5_k.gguf	2.85 GiB	Balanced K-family profile.
yue2-3b-q5_k_m.gguf	2.96 GiB	Audio.cpp-native mixed profile: Q5_K base with selected AR/NAR value and down projections in Q6_K; not a byte-identical llama.cpp preset.
yue2-3b-q5_1.gguf	3.04 GiB	Legacy-compatible Q5 profile.
yue2-3b-q5_0.gguf	2.85 GiB	Legacy-compatible Q5 profile.
yue2-3b-q4_k.gguf	2.48 GiB	Balanced K-family profile.
yue2-3b-q4_k_m.gguf	2.68 GiB	Audio.cpp-native mixed profile: Q4_K base with selected AR/NAR value and down projections in Q6_K; not a byte-identical llama.cpp preset.
yue2-3b-q4_1.gguf	2.67 GiB	Legacy-compatible Q4 profile.
yue2-3b-q4_0.gguf	2.48 GiB	Direct Q4_0 compatibility profile.
yue2-3b-q3_k.gguf	2.09 GiB	Small K-family profile.
yue2-3b-q3_k_l.gguf	2.39 GiB	Audio.cpp-native mixed small profile: Q3_K base with selected AR/NAR value and down projections in Q6_K; not a byte-identical llama.cpp preset.
yue2-3b-q3_k_m.gguf	2.29 GiB	Audio.cpp-native mixed profile: Q3_K base with selected AR/NAR value and down projections in Q5_K.
yue2-3b-q3_k_s.gguf	2.19 GiB	Audio.cpp-native compact mixed profile: Q3_K base with selected AR/NAR value and down projections in Q4_K.
yue2-3b-q2_k.gguf	1.78 GiB	Smallest published main profile.
yue2-3b-q2_k_s.gguf	1.96 GiB	Audio.cpp-native smallest mixed profile: Q2_K base with selected AR/NAR value and down projections in Q4_K; not a byte-identical llama.cpp preset.
yue2-vae-f16.gguf	0.25 GiB	Default VAE decoder; pair with one main file.
yue2-vae-f32.gguf	0.49 GiB	F32 VAE reference decoder.
sidecars/	—	Configs and tokenizer required by the native loader.
examples/	—	Official requests, ABC scores, and verified audio demos.
reference/official-python/	—	Pinned upstream Python wheel, source, docs, and licenses.

Recommended quantized profiles

These are three practical starting points, all excluding the BF16 reference:

Profile	Choose it when
yue2-3b-q8_0.gguf	You want the closest practical profile to the BF16 reference.
yue2-3b-q4_k_m.gguf	You want the main size/quality balance.
yue2-3b-q2_k_s.gguf	You need the smallest mixed profile that passed runtime smoke.

Q8_0 is the default main profile in the example below. Select another main file with --session-option yue2.model_gguf=<file>. Use yue2-vae-f16.gguf unless you specifically want the F32 decoder. All published main files were quantized directly from the BF16 reference; no quantized file was used as the source for another quantization.

The Q8_0_F16, Q6_K_M, Q3_K_M, and Q3_K_S files are audio.cpp-native mixed profiles. Their suffixes describe the higher-precision tensors selected for the AR/NAR path; they are not byte-identical llama.cpp preset names.

For a first download, Q8_0 + F16 is the recommended balance. Q4_0 is included for compatibility with existing audio.cpp workflows; Q4_K is a separate K-family artifact, not a renamed copy. Q4_K_M is an audio.cpp-native mixed profile that spends extra bits on selected AR/NAR projections; it is named by its target tier and is not a byte-identical llama.cpp preset. Q5_K_M uses the same projection-preservation strategy at a higher base precision. Q3_K_L preserves those projections at Q6_K for a smaller base profile. Q3_K is the safer small profile for a long generation. Q2_K is useful when storage is tight. Q2_K_S keeps the same projection-preservation idea while using Q4_K for those 112 tensors; it is a compact runtime option, not a quality claim. The published longform Q2_K runtime check reached the configured semantic-token limit; see the measured table below.

Quick start

Build or obtain the dev branch of audio.cpp, then point --model at this downloaded model directory:

<AUDIOCPP>/build/debug/bin/audiocpp_cli \
  --task gen \
  --family yue2 \
  --model <YUE2_GGUF> \
  --backend cuda \
  --threads 8 \
  --lyrics $'[Verse]\nSoft morning light is touching the window.\nI hear the city waking below.\n[Chorus]\nStay with the rhythm, let it carry us home.\nSing with the sunrise, we are never alone.' \
  --request-option 'style=English, indie pop, bright acoustic guitar, soft drums, warm lead vocal' \
  --request-option cot=off \
  --seed 831001 \
  --out yue2.wav \
  --log --metrics

<YUE2_GGUF> must be the directory containing the GGUF files and sidecars/, not just the path to one main GGUF. The command loads yue2-3b-q8_0.gguf and yue2-vae-f16.gguf by default. For example, to use Q4_0 and make the VAE choice explicit:

--session-option yue2.model_gguf=yue2-3b-q4_0.gguf \
--session-option yue2.vae_gguf=yue2-vae-f16.gguf

The upstream model supports optional symbolic planning with cot=melody or cot=full; see the audio.cpp YuE2 documentation for ABC conditioning and advanced generation options.

Verified audio demos

These demos use the upstream examples/tonight-awake.json request, cot=full, eight NAR steps, seed 12300, and the audio.cpp CUDA runtime. They are 48 kHz stereo WAV files generated from this package; the BF16 file is a reference, not a claim that every quant has identical quality.

Profile	Decoder	Demo
BF16	F32	tonight-awake-bf16-f32vae.wav
Q8_0	F16	tonight-awake-q8_0.wav
Q4_0	F16	tonight-awake-q4_0.wav

The full official Python examples and ABC inputs are in examples/README.md. They cover direct generation, melody conditioning, and edited harmony. For the upstream Python API and an exact installable wheel, see reference/official-python/README.md.

Measured runtime

The following is a runtime smoke/longform check, not a quality benchmark. It was measured on one NVIDIA A10M GPU (19,190 MiB), audio.cpp commit 6fbbee4efd1c2251d3774edd89a002a0039da5f7, CUDA backend, eight CPU threads, the official tonight-awake.json request, cot=full, eight NAR steps, and seed 12300. RTF is wall time divided by generated audio duration; lower is faster. truncated reports the runtime's semantic-token limit status.

Main + VAE	Audio	Wall	RTF	Peak GPU0	Truncated
BF16 + F32	224.799 s	144.9 s	0.6446	11,095 MiB	no
Q8_0 + F16	218.039 s	112.2 s	0.5145	8,033 MiB	no
Q4_0 + F16	219.999 s	96.3 s	0.4379	6,441 MiB	no
Q2_K + F16	359.999 s	153.4 s	0.4261	7,261 MiB	yes

The compact reproducible summary is in reproducibility/benchmark.csv. It should not be compared directly with results from a different GPU, decoder, runtime build, or generation setting.

Reproducibility and validation

The complete source, converter, command, and validation provenance is in reproducibility/manifest.md. Verify the downloaded files with SHA256SUMS.txt.

The main GGUF files were converted with audio.cpp's native YuE2 converter at commit 6fbbee4efd1c2251d3774edd89a002a0039da5f7. Every main type in the table was produced independently and directly from the BF16 main GGUF. The audio.cpp converter does not accept an imatrix, and llama.cpp's generic imatrix tool cannot load the audiocpp architecture, so no imatrix is claimed or bundled for this release.

Every published main variant was load-tested and used for a short CUDA generation run with the F16 VAE. Each run produced a non-empty 48 kHz stereo WAV. The F32 VAE also passed a CUDA smoke run with Q8_0. These are runtime smoke tests, not a fixed hold-out fidelity evaluation; no fidelity table is included.

The official Python reference is pinned separately from the GGUF conversion: its source snapshot and wheel hash are recorded in reference/official-python/README.md.

License and attribution

The upstream YuE2-3B and YuE2 VAE checkpoint weights are released under Creative Commons Attribution-NonCommercial 4.0 International. The license text is included in LICENSE, and additional bundled notices are in THIRD_PARTY_NOTICES.md.

The bundled upstream Python source is Apache 2.0; its code license and model weight license are preserved under reference/official-python/. Third-party code notices remain separate from the YuE2 weight license. Do not use the model weights commercially unless you have separate permission from the rights holders.

These are community GGUF quantizations, not an official YuE2 release or endorsement.