VoxCPM-Kernel/VoxCPM2-Arabic-Dialects-Distilled-DiT

VoxCPM2 Arabic Dialects: distilled LocDiT (3 and 2 steps)

Few-step replacements for the diffusion decoder (LocDiT) of VoxCPM2-Arabic-Dialects. They are distilled from that model's own 10-step decoder on Arabic speech.

• student_k3.pt (3 steps) has been the Turbo engine's default since 2026-10-17. It sounds more natural than the 2-step student, with the same CER.
• student_k2_dmd.pt (2 steps) is 11 ms faster at 200 requests.

At a glance

For	VoxCPM2-Arabic-Dialects (all 11 adapters)
Teacher	the same model's original LocDiT: 10 Euler steps, CFG 2.0, each clip through its dialect's adapter
Students	3 or 2 steps, no CFG, 12 layers, 212M parameters
Changed weights	LocDiT only (feat_decoder.estimator.*); the adapters' LocDiT LoRAs still apply on top
Training data	1.48M teacher patches on VoxCPM2-Arabic-Train-v2 contexts (135 h); evaluation speakers and texts left out
Engine default	student_k3.pt
License	CC BY-NC-SA 4.0 (non-commercial, like its training data)

Files

File	Steps	Training	Use it for
student_k3.pt	3	from student_k2_dmd.pt, spectral loss, 1,000 steps	Default
student_k2_dmd.pt	2	MSE → spectral → DMD (distribution matching)	Faster option
student_k2_spectral.pt	2	MSE → spectral	Comparison
student_k2_mse.pt	2	MSE to the teacher's patches	Comparison

Results

Held-out evaluation set: 17 dialects, voice cloning from another clip of the same speaker, int8 engine. The table uses the 980 clips (490 sentences × 2 seeds) that were not used to choose student_k3.pt's checkpoint.

decoder	steps	first audio p95, 200 requests	CER	WER	UTMOS	DNSMOS	speaker sim
original LocDiT (teacher)	10 + CFG	270 ms	5.28 %	15.2 %	2.41	3.00	0.941
student_k3.pt	3	103 ms	5.34 %	15.7 %	2.24	2.94	0.939
student_k2_dmd.pt	2	92 ms	5.27 %	15.3 %	2.15	2.82	0.933

• CER / WER: error rate of the better of two speech recognizers (Whisper large-v3, Cohere Transcribe Arabic). Lower is better.
• UTMOS / DNSMOS: predicted naturalness and quality, 1-5. Higher is better.
• 3 steps against 2 (paired bootstrap, 95 %):
  • UTMOS +0.09 [+0.07, +0.11], DNSMOS +0.12 [+0.10, +0.14] and speaker similarity +0.006: all beyond noise;
  • CER and WER the same within noise.
• Gap to the 10-step teacher (UTMOS): −0.26 with 2 steps, −0.17 with 3.
• Hindi-English (hindi_english adapter, 4 voices × 20 sentences × 2 seeds, Whisper large-v3 in Hindi): UTMOS +0.14 [+0.08, +0.19] with 3 steps; CER and WER the same within noise.
• Speed (one A10, 200 voice-cloned requests at once): first audio 92 → 103 ms p95, whole sentence 1.94 → 2.17 s. A single request takes about 1 ms longer.

Per dialect (60 clips each, 40 for Iraqi and Moroccan, so small differences are noise):

dialect	CER, 10-step	CER, 2-step	CER, 3-step	UTMOS, 10-step	UTMOS, 2-step	UTMOS, 3-step
MSA	2.8 %	2.7 %	2.8 %	2.08	1.98	1.99
Egyptian	1.1 %	1.4 %	1.4 %	2.70	2.43	2.67
Palestinian	1.6 %	1.8 %	2.0 %	2.71	2.22	2.66
Lebanese	5.9 %	6.3 %	5.6 %	2.85	2.61	2.66
Syrian	6.9 %	5.8 %	6.5 %	2.37	2.33	2.35
Iraqi	4.2 %	3.5 %	3.1 %	2.40	2.16	2.41
Saudi	3.0 %	3.0 %	3.2 %	2.32	2.13	2.20
Najdi	8.5 %	6.5 %	7.7 %	2.90	2.44	2.50
Hejazi	3.7 %	3.9 %	4.5 %	2.49	2.07	2.24
Gulf	13.8 %	14.2 %	13.6 %	1.32	1.37	1.34
Bahraini	4.6 %	5.0 %	3.7 %	2.59	2.22	2.39
Yemeni	4.2 %	4.8 %	4.2 %	2.55	2.31	2.37
Sudanese	4.1 %	4.7 %	4.6 %	2.47	2.20	2.21
Libyan	5.7 %	6.0 %	5.9 %	2.61	2.34	2.36
Tunisian	7.0 %	7.1 %	8.8 %	2.15	2.10	1.99
Algerian	5.7 %	5.5 %	5.2 %	2.07	1.55	1.62
Moroccan	7.3 %	7.5 %	8.2 %	2.33	2.10	2.11

3 steps against 2, beyond noise:

• Better: Palestinian (UTMOS +0.44), Iraqi (+0.25), Egyptian (+0.24), Bahraini (+0.17), Hejazi (+0.17), Saudi (+0.07) and Algerian (+0.07).
• Worse: Tunisian (UTMOS −0.11, CER +1.7).

Usage

The Turbo engine loads student_k3.pt by default:

tts = await DialectTTS.load("VoxCPM-Kernel/VoxCPM2-Arabic-Dialects")   # 3 steps, the default
tts = await DialectTTS.load("VoxCPM-Kernel/VoxCPM2-Arabic-Dialects",
                            dit="VoxCPM-Kernel/VoxCPM2-Arabic-Dialects-Distilled-DiT:student_k2_dmd.pt")  # 2 steps

HTTP server: --dit VoxCPM-Kernel/VoxCPM2-Arabic-Dialects-Distilled-DiT:student_k3.pt (or :student_k2_dmd.pt).

A checkpoint holds the LocDiT weights plus its sampler settings (steps, schedule, mean_mode). The engine reads them, keeps CUDA graphs and int8, and needs no other change.

Training

	student_k3.pt	student_k2_dmd.pt
Start	student_k2_dmd.pt	the English student (VoxCPM2-Turbo-distilled-DiT)
Grid (t)	1 → 0.7071 → 0.4239 → 0	1 → 0.7071 → 0
Stages	spectral 1,000 steps (1 h 30 min)	MSE 4,000 → spectral 1,500 → DMD 1,500 steps (12 h)
Selection	UTMOS on a development subset (170 sentences, another seed), with CER and runaway guards; a DMD stage after it lowered UTMOS and was not used	best validation loss on 20k held-out patches
Teacher data	the same 1.5M patches	1.5M patches: the conditioning, the noise and the teacher's 10-step patch for each target patch
Hardware	one NVIDIA A10 (24 GB)	one NVIDIA A10 (24 GB), shared with another training run

Code: Speech-Next/voxcpm2-arabic-training.

• student_k3.pt: scripts/run_student_k3.sh.
• student_k2_dmd.pt: scripts/run_dit_adapters.sh and arabic_voxcpm/distill/dit.py.

The branch adapter-lora-experiment keeps a failed experiment, per-adapter LoRAs for the 2-step student. It is not for use.

License

CC BY-NC-SA 4.0: the training contexts come from non-commercial data (SADA, VoxTube reference voices). The base model VoxCPM2 is Apache-2.0.