VoxCPM-Kernel/VoxCPM2-Arabic-Teacher-v2

VoxCPM2 Arabic Teacher v2

One VoxCPM2 model for 17 Arabic dialects; a dialect tag in front of the text picks the dialect. It is replaced by Teacher-v3, which is better on CER, WER and naturalness. Use this one only to reproduce earlier results.

At a glance

Base model	openbmb/VoxCPM2 (2B parameters, 48 kHz)
Trained	LoRA rank 64 on base_lm (attention and FFN) + 17 dialect tag embeddings: 67M parameters, merged into the weights
Checkpoint	step 2500 of 3000 (best validation)
Dialects	17: msa, egyptian, palestinian, lebanese, syrian, iraqi, saudi, najdi, hejazi, gulf, bahraini, yemeni, sudanese, libyan, tunisian, algerian, moroccan
Aliases	emirati, qatari, khaliji → gulf; jordanian, levantine → palestinian
Training data	135 h: 103 h real speech, 32 h synthetic
Size	5.23 GB
License	CC BY-NC-SA 4.0 (non-commercial)
Recommended instead	Teacher-v3

Files

path	what it is
model.safetensors, audiovae.pth, config.json	the merged model, loaded as a normal VoxCPM2 checkpoint
tokenizer.json, tokenizer_config.json, special_tokens_map.json, chat_template.jinja	the tokenizer, with the dialect tags as special tokens
dialects.json	dialect → tag token, and the aliases
lora/trainable.safetensors, lora/train_state.json	the unmerged LoRA and tag embeddings, and the training settings

Results

Held-out evaluation set: 17 dialects × 30 sentences × 2 seeds = 1020 clips. Its speakers and texts are not in the training data.

model	CER	WER	UTMOS	speaker sim	dialect region
VoxCPM2-Arabic-Dialects (11 LoRA adapters)	4.9 %	15.0 %	2.48	0.94	64 %
Teacher-v2 (this model)	4.8 %	14.6 %	2.35	0.95	64 %

dialect	CER, Dialects	CER, v2	UTMOS, Dialects	UTMOS, v2
msa	3.0 %	2.7 %	2.15	2.10
egyptian	1.6 %	1.5 %	2.76	2.84
palestinian	3.0 %	2.5 %	2.82	2.86
lebanese	7.1 %	7.8 %	2.89	2.92
syrian	7.1 %	6.1 %	2.46	2.42
iraqi	3.2 %	3.6 %	2.44	2.30
saudi	3.2 %	4.4 %	2.35	2.20
najdi	4.2 %	3.7 %	3.02	2.32
hejazi	2.9 %	4.4 %	2.69	1.82
gulf	7.8 %	7.3 %	1.74	1.30
bahraini	3.7 %	4.9 %	2.70	2.52
yemeni	5.4 %	4.9 %	2.60	2.63
sudanese	4.0 %	3.7 %	2.43	2.42
libyan	5.5 %	5.0 %	2.61	2.74
tunisian	9.9 %	7.7 %	2.15	2.30
algerian	5.6 %	5.5 %	2.17	2.08
moroccan	7.0 %	6.6 %	2.21	2.20

• CER / WER: character / word error rate of a speech recognizer's transcript (better of Whisper large-v3 and Cohere Transcribe Arabic); lower is better. UTMOS: predicted naturalness, 1-5. Speaker sim: similarity to the reference voice, 0-1. Dialect region: share placed in the right region by a dialect classifier (ADI-5).
• 60 clips per dialect, so small differences are noise. Beyond noise (95 %): CER better in tunisian and syrian, worse in hejazi, saudi and bahraini.

On the larger evaluation set used for Teacher-v3 (1320 clips), Teacher-v3 scores CER 4.34 % vs 5.03 %, WER 13.6 % vs 15.1 % and UTMOS 2.49 vs 2.35.

Usage

Install the Turbo engine (./setup.sh), then:

import asyncio
from nanovllm_voxcpm import DialectTTS

async def main():
    tts = await DialectTTS.load("VoxCPM-Kernel/VoxCPM2-Arabic-Teacher-v2", dit=None, w8a8=False)
    await tts.add_voice("speaker", "reference.wav")
    audio = await tts.synthesize("هلا والله، كيف حالك اليوم؟", dialect="najdi", voice="speaker")  # 48 kHz
    await tts.close()

if __name__ == "__main__":
    asyncio.run(main())

• dit=None: the model's own LocDiT (10 steps). A distilled 2-step LocDiT for this model is not made yet.
• w8a8=False: run in bf16; int8 is not calibrated for this model.

Training data

135 h. Real speech, 103 h: Rabe3/lahgtna-arabic-tts-24khz and SADA, kept only with CER ≤ 10 % and DNSMOS ≥ 3.0. Synthetic speech, 32 h: the earlier dialect adapters and Habibi-TTS, kept only with CER ≤ 8 % and DNSMOS ≥ 3.0. Published as VoxCPM-Kernel/VoxCPM2-Arabic-Train-v2.

License

CC BY-NC-SA 4.0, because some training data is non-commercial (SADA, VoxTube reference voices, Habibi-TTS non-commercial checkpoints). The base model VoxCPM2 is Apache-2.0.