Fun-CosyVoice3-0.5B-ONNX-WebGPU

FunAudioLLM/Fun-CosyVoice3-0.5B-2512 as ONNX graphs for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference: plain text-to-speech with the upstream's default voice prompt.

Changes from the original

Converted from the upstream PyTorch code (FunAudioLLM/CosyVoice) into separate graphs by tools/export/export_cosyvoice3.py in Quick Inference (in float32 they reproduce the PyTorch pipeline: same tokens, mel cosine 0.9995), then quantized without calibration (round to nearest, symmetric, blocks of 32):

• onnx/embed_q4.onnx: the text and speech-token embedding table, 4-bit (GatherBlockQuantized).
• onnx/llm_q8.onnx: the Qwen2 0.5B language model (ONNX Runtime GenAI's model builder graph, inputs_embeds input, float32 KV cache), 8-bit MatMulNBits.
• onnx/flow_encoder_q4.onnx: speech tokens, speaker embedding and prompt mel → the flow's conditions, 4-bit.
• onnx/flow_step_q8.onnx: one Euler step of the DiT flow matcher with classifier-free guidance (0.7), 8-bit.
• onnx/hift_q4c8.onnx: the HiFT vocoder, 4-bit matrix multiplications and int8 convolutions; its noise is an input.
• voice_default.json: the default voice, precomputed with the upstream's own frontend (PyTorch speech tokenizer and CAM++) from a clearly licensed clip: prompt text ids, 109 speech tokens, the speaker embedding and the prompt mel, so the speech tokenizer and CAM++ aren't needed. The clip is utterance 4446_2275_000002_000009 (speaker 4446, 4.4 s, test-clean) of LibriTTS-R (Koizumi et al., 2023; via mythicinfinity/libritts_r), licensed CC BY 4.0, a restored version of LibriTTS, which is made from public-domain LibriVox audiobook recordings; its transcript: "I felt it in my bones when I woke this morning that something splendid was going to turn up." Only the derived features are included, not the audio.
• flow_noise_f16.bin: the upstream's fixed flow start noise (float16 [80, 3000]).
• tokenizer.json: the Qwen2 tokenizer with CosyVoice 3's special tokens.

These are modified files: their outputs are close to, but not identical to, the original model's. On eight English test sentences, Whisper small.en heard it with an 11.4% word error rate (12.5% in Chrome) against 13.6% for the original PyTorch pipeline, all with the upstream's own prompt clip; with the default voice here it was 12.5% in Chrome. It samples, so single runs vary.

Use

Per chunk of up to 80 text tokens: the language model reads [SOS, prompt text ids, text ids, TASK, prompt speech tokens] and samples speech tokens (repetition-aware sampling: top-p 0.8, top-k 25, window 10) until a stop token (at least 2 and at most 20 per text token); the flow encoder and ten cosine-schedule flow steps from the fixed noise make the mel of prompt and speech; the prompt's frames are dropped; HiFT turns the rest into 24 kHz audio with uniform noise. Text in English is normalized first (numbers, dates, times and symbols in words, as upstream's wetext does).

Licence

Apache 2.0, as Fun-CosyVoice 3 (LICENSE). The default voice (voice_default.json) is derived from LibriTTS-R, CC BY 4.0 (attribution above).