BricksDisplay/BreezyVoice-GGUF

BreezyVoice GGUF

GGUF conversion of MediaTek-Research/BreezyVoice, a Taiwanese-Mandarin fine-tune of FunAudioLLM/CosyVoice-300M, for codec.cpp and llama.rn.

BreezyVoice ships its own speech LM (llm.pt). Its flow decoder, HiFT vocoder, CAMPPlus speaker encoder and S3 speech tokenizer are byte-identical to CosyVoice-300M's. The model is split into two files:

• Speech LM (breezyvoice-lm-<quant>.gguf): the 14-layer WeNet Transformer with relative-position attention, as llama.cpp arch cosyvoice, with the Whisper multilingual vocabulary. Stock llama.cpp does not have this arch; it needs codec.cpp's patches/cosyvoice-llamacpp.patch (llama.rn carries it).
• Codec + codec_lm (breezyvoice-codec-<quant>.gguf): the conformer + length regulator + U-Net CFM flow, HiFT vocoder (22.05 kHz), the LM adaptor (text encoder, speech embedding, speech head), the builtin voices, and the speaker encoders used for voice cloning.

Files

Speech LM

File	Size
breezyvoice-lm-f16.gguf	397 MB
breezyvoice-lm-q8_0.gguf	238 MB
breezyvoice-lm-q4_k_m.gguf	161 MB

The relative-position tensors (attn_pos*) stay F32 in the quantized files: they are added to activations, and quantized operands are not supported there.

Speech LM check (llama.rn, CPU, builtin voice zh_female, F16 codec, n_timesteps=4, one sentence, seeds 1–12, Whisper small): mean CER F16 0.146 (6/12 exact), Q8_0 0.240 (2/12), Q4_K_M 0.109 (3/12). BreezyVoice's top-k sampling varies a lot between seeds, so these numbers are noisy; Q8_0 dropped the first clause in two runs.

Codec + codec_lm

File	Size
breezyvoice-codec-f32.gguf	1.42 GB
breezyvoice-codec-f16.gguf	753 MB
breezyvoice-codec-q8_0.gguf	405 MB
breezyvoice-codec-q5_k_m.gguf	316 MB
breezyvoice-codec-q4_k_m.gguf	287 MB

The quantized codec files need codec.cpp 0289fcc or newer (quantized conv layouts and their kernel sizes). The f16 / f32 files also work with 360163d. llama.rn vendors 0289fcc on its chore/bump-codec-cpp branch.

Quantized codec recipe (scripts/quantize-gguf.py): the decoder at Q8_0 / Q5_K / Q4_K with Q8_0 fallback, the LM adaptor (lm.*) at Q8_0 in every file, the S3 tokenizer codebook kept F32.

Codec check (codec.cpp tts-cli, Whisper CER, CPU and Vulkan, one zh sentence): f32 / f16 0.2 (a trailing 嗯), q8_0 / q5_k_m 0.133, q4_k_m 0.333 on CPU (one word) and 0.133 on Vulkan.

Voices

• Builtin voices from spk2info.pt: zh_female (default), zh_male, ja_male, yue_female, en_female, en_male, ko_female. They carry prompt tokens and mel but no transcript, so they run in cross-lingual mode.
• Voice cloning from reference audio: with the clip's transcript it runs zero-shot (the reference speech tokens are part of the LM prompt, 50 per second of audio); without a transcript, cross-lingual.

Text input

• Upstream BreezyVoice annotates Mandarin with 注音 (bopomofo) via g2pw before synthesis. That step is not part of these files: plain text works, but polyphonic characters may be misread. Add the annotations yourself for the best results.
• Text normalization and sentence splitting are not ported; pass normalized sentences.

Defaults and tuning

Inference settings are GGUF metadata in the codec file, overridable at load (codec.cpp --override-kv KEY=TYPE:VALUE, llama.rn initVocoder({ kv_overrides })):

Setting	Key	Default
Flow Euler steps	cosyvoice.flow.n_timesteps	10
Flow CFG	cosyvoice.flow.cfg_rate	0.7
Speaking rate	cosyvoice.flow.speed	1.0
LM sampling	codec.lm.cosyvoice.sampling	top_k (BreezyVoice)
Top-k	codec.lm.cosyvoice.top_k	25
Min / max speech tokens per text token	codec.lm.cosyvoice.min_token_text_ratio / max_token_text_ratio	2 / 20

Flow decoding is n_timesteps × 2 U-Net passes and dominates the run time. On phones, cosyvoice.flow.n_timesteps=int:4–6 is a good trade.

Usage

codec.cpp:

tts-cli synthesize --model breezyvoice-codec-f16.gguf --backbone breezyvoice-lm-f16.gguf \
    --voice zh_female --text "今天天氣真好，我們一起去公園走走吧。" --output out.wav
# zero-shot voice clone
tts-cli synthesize ... --ref-audio ref.wav --ref-text "參考音訊的逐字稿" --text "..." --output out.wav

llama.rn: load the speech LM with embedding: true and an F16 KV cache (its relative positions do not run under flash attention, which a quantized V cache forces), attach the codec with initVocoder, and pass a voice name or a LlamaSpeaker as speaker.

License

Apache-2.0, as BreezyVoice and CosyVoice-300M.