VoxCPM2-Flash-GGUF

audio.cpp GGUF of the Jarvis-style Khmer / English VoxCPM2 fine-tune from nphearum/VoxCPM2-Flash: base openbmb/VoxCPM2 with the LoRA merged into the weights.

Use num_inference_steps: 5. It is ~1.45x faster than the default 10 steps and stays closest to full quality (measured on an RTX 3090, see below).

file	LM weights	size	ASR WER	RTF
voxcpm2-q8_0.gguf	Q8_0 (same layout as audio.cpp's official VoxCPM2 Q8_0)	2.96 GB	1.1%	0.213
voxcpm2-q6_k.gguf	Q6_K	2.54 GB	1.1%	0.188
voxcpm2-q5_k_m.gguf	Q5_K, attn V / MLP down Q6_K	2.38 GB	1.1%	0.191
voxcpm2-q4_k_m.gguf	Q4_K, attn V / MLP down Q6_K	2.23 GB	0.0%	0.212
voxcpm2-q4_0.gguf	Q4_0	2.11 GB	0.0%	0.183

Only the two language models (~1.7B of ~2.3B params) are quantized below Q8_0. The local DiT, feature encoder and projections stay Q8_0, the token embedding F16 and the AudioVAE F32, which is why the files stop shrinking around 2.1 GB. All tensors were quantized with ggml's own ggml_quantize_chunk.

WER: 8 unseen English lines x 2 seeds at num_inference_steps: 5, transcribed with Qwen3-ASR-0.6B (audio.cpp); the remaining "errors" are the ASR spelling Penompen for Phnom Penh. WER measures intelligibility, not voice similarity, so listen before picking a small file. RTF on an RTX 3090 with the ASR model co-resident; the LMs are compute-bound at batch 1 here, so lower bits mainly save memory, not time.

Usage

Download one file into its own folder (audio.cpp loads a folder holding a single .gguf):

hf download nphearum/VoxCPM2-Flash-GGUF voxcpm2-q4_k_m.gguf --local-dir /models/voxcpm2-jarvis

Point audio.cpp at the folder:

{"models": [{"id": "voxcpm2-jarvis", "family": "voxcpm2", "path": "/models/voxcpm2-jarvis"}]}

audiocpp_server --config server.json --port 8080
curl http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' -d '{
  "model": "voxcpm2-jarvis",
  "input": "Good evening, sir. ការប្រជុំថ្ងៃស្អែកត្រូវបានលុបចោលហើយ។",
  "num_inference_steps": 5,
  "response_format": "wav"
}' -o out.wav

Speed (audio.cpp, CUDA, RTX 3090, Q8_0)

num_inference_steps	guidance_scale	RTF	DiT calls
10 (default)	2	0.242	9
5	2	0.167	4
2	2	0.130	1 (quality drops sharply)

The base LMs take ~0.12 RTF on their own, so fewer diffusion steps cannot go much below that.

Why no Flash head here?

The PyTorch repo ships a distilled few-step "Flash" diffusion head. audio.cpp's VoxCPM2 sampler always feeds the DiT dt = 0, zeroes the first flow step, and runs the CFG batch even at guidance_scale: 1 (all verified with probe GGUFs), so the Flash head can't run there and a stock-sampler re-distill gains no speed over simply using 5 steps. The weights in this repo use the original diffusion head.

Training data

~45 synthetic clips (Khmer, English and code-switched assistant lines) generated by voice-cloning one reference recording with VoxCPM2. Small dataset: expect the voice to generalise imperfectly.