VoxCPM2 Arabic Dialects, 2:4 sparse
VoxCPM2-Arabic-Dialects with most of its main language model's feed-forward layers pruned to the 2:4 pattern (2 of every 4 weights are zero) and then trained back to the dense model's outputs. The Turbo engine runs these layers on the GPU's sparse tensor cores, so the first audio comes sooner. The 11 dialect adapters are the dense model's.
Status: experimental. First audio is about 8 ms sooner at 200 requests. Overall WER is 0.5 points higher, about the evaluation's run-to-run noise, but two groups lose for real: Khaliji (WER +3.5 to +4.6) and Sudanese (WER +2 to +2.8). Use VoxCPM2-Arabic-Dialects unless you need the speed.
At a glance
| Base | VoxCPM2-Arabic-Dialects (VoxCPM2, 2B parameters, 48 kHz) with its 11 adapters |
| Changed weights | base_lm's FFN (gate, up, down) in 24 of its 28 layers: 2:4 sparse along the input; layers 24-27 keep their dense FFN; everything else unchanged |
| Pruning | SparseGPT 2:4, calibrated on VoxCPM2-Arabic-Train-v2 contexts with every adapter |
| Recovery | 2,500 steps of distillation from the dense model with the same adapter |
| Engine | the sparse int8 kernels are in the Turbo engine from PR #21 on; it detects the 2:4 weights itself (an older engine runs this model as a dense one, with no speed gain) |
| License | CC BY-NC-SA 4.0 (trained on non-commercial data); VoxCPM2 itself is Apache-2.0 |
Revisions
| branch | what | CER % | WER % | late stop % | runaway % |
|---|---|---|---|---|---|
| (dense model) | VoxCPM2-Arabic-Dialects | 5.24 | 15.54 | 3.48 | 0.45 |
r1 |
all 28 layers sparse, 2,000 recovery steps | 5.43 | 16.05 | 3.86 | 0.91 |
r2 |
as r1, stop-loss weight 10, 3,000 steps | 5.39 | 15.94 | 4.55 | 0.68 |
main (r3) |
layers 24-27 dense, 2,500 steps | 5.45 | 16.07 | 3.71 | 1.06 |
Results (r3)
Held-out evaluation set: 17 dialects ร 40 sentences ร 2 seeds (Iraqi and Moroccan: 30), 1,320 clips, voice cloning from another clip of the same speaker, int8 engine, the default distilled LocDiT.
| model | CER % | WER % | UTMOS | speaker sim | late stop % | runaway % |
|---|---|---|---|---|---|---|
| dense | 5.24 | 15.54 | 2.14 | 0.933 | 3.48 | 0.45 |
| 2:4 sparse (this model) | 5.45 | 16.07 | 2.15 | 0.934 | 3.71 | 1.06 |
Time to first audio, one A10, all requests sent at once (average / p95 of 2 runs each, dense and sparse alternating):
| requests at once | dense | 2:4 sparse |
|---|---|---|
| 1 | 25 / 30 ms | 22 / 23 ms |
| 8 | 26 / 27 ms | 24 / 24 ms |
| 64 | 36 / 38 ms | 35 / 36 ms |
| 200 | 93 / 94 ms | 85 / 86 ms |
- CER / WER: the better of Whisper large-v3 and Cohere Transcribe Arabic; lower is better. UTMOS: predicted naturalness, 1-5.
Usage
from nanovllm_voxcpm import DialectTTS
tts = await DialectTTS.load("VoxCPM-Kernel/VoxCPM2-Arabic-Dialects-Sparse24")
await tts.add_voice("ahmed", "ahmed.wav")
audio = await tts.synthesize("ููุง ูุงูููุ ููู ุญุงูู ุงูููู
ุ", dialect="najdi", voice="ahmed")
The engine logs 24 with a 2:4 sparse FFN when it uses the sparse kernels (an Ampere or newer GPU).
Training
| Data | contexts of VoxCPM2-Arabic-Train-v2 (135 h); evaluation speakers and texts left out; all 11 adapters equally likely |
| Calibration | SparseGPT, layer by layer, gate / up then down |
| Targets | the dense model's LocDiT conditioning, stop logits and FSQ latents on 1,500,000 target patches |
| Recovery | only the kept FFN weights train (the zeros stay), AdamW, 2,500 steps |
| Hardware | one NVIDIA A10 |
| Code | Speech-Next/voxcpm2-arabic-training: arabic_voxcpm/distill/sparse_ffn.py, scripts/run_sparse24.sh |
- Downloads last month
- 22