waxal-benchmarking/voxcpm2-top10-full

WAXAL VoxCPM2 — Top-10 African TTS

A full fine-tune of VoxCPM2 (2B params, tokenizer-free, diffusion-AR TTS with voice cloning) on the 10 highest-resource languages in the WAXAL/AfroNet collection, produced by the WAXAL project at LyngualLabs.

This is a research checkpoint. It is not production-polished — see Known Limitations below before using it for anything beyond experimentation.

This model is trained on a cleaned version of the WAXAL/AfroNet pooled corpus (see Data Cleaning below), the same methodology used for its sibling models. It's the middle of three sibling models (Top-5 / Top-10 / all-28) trained from the same cleaned data, to compare whether language count or data quality matters more at this scale.

Languages

VoxCPM2 has no built-in mechanism for specifying language at inference — the base model infers pronunciation purely from text orthography. We use a dedicated <lang:XX> tag per language, registered as a real token in the tokenizer and resized into the text embedding table (mean-initialized, then learned during fine-tuning). The tag is mandatory — always prepend it to the text you want synthesized, e.g.:

<lang:yo> Ẹ káàárọ̀! Báwo ni oorun rí lálẹ́ àná?

Language	Tag	Hours after cleaning	Hours after resampling	Retention after cleaning
Swahili	<lang:sw>	922.8	689.9	96.4%
Kinyarwanda	<lang:rw>	910.5	685.3	96.1%
Yoruba	<lang:yo>	732.3	614.5	74.0%
Igbo	<lang:ig>	620.2	565.6	69.6%
Hausa	<lang:ha>	534.2	524.9	82.0%
Dholuo	<lang:luo>	165.1	291.8	95.4%
Wolaytta	<lang:wal>	141.1	269.7	78.1%
Amharic	<lang:am>	154.5	282.3	86.6%
Oromo	<lang:om>	151.1	279.1	85.7%
Sidama	<lang:sid>	140.4	269.1	79.6%

Source data: the AfroNet speech collection (Professor/<language>-speech-data), after the Phase 1 cleaning pass described below. This checkpoint's base, VoxCPM2_langtagged, has exactly these 10 tags registered — no unused tag rows, unlike the Top-5 sibling model.

Data Cleaning

Earlier releases in this collection (voxcpm2-multilang28-full, voxcpm2-multilang10-full, voxcpm2-bantu8-full) trained directly on the pooled raw corpus. This model and its Top-5 / all-28 siblings are trained on data that first passed through a dedicated cleaning pipeline, applied per language before any resampling:

1. Clip/peak check — reject clips with a sustained near-full-scale amplitude (likely clipped/distorted audio).
2. Silence trim — trim leading/trailing silence before any downstream check, so later checks evaluate the audio that's actually used for training.
3. Duration-vs-text-length ratio check — flag clips whose trimmed duration is a statistical outlier relative to their text length for that language. This caught a real upstream data bug: a small number of rows had their text field corrupted with concatenated Common Voice metadata (worst case, 225,940 characters for a single 9-second clip in the Luganda corpus).
4. ASR-CER gate — transcribe the trimmed audio with a small ASR model (omniASR_CTC_1B) and compare against the label text (CER threshold 0.3, diacritic/tone-mark-insensitive comparison so correct transcriptions aren't penalized for missing a tone mark the ASR doesn't produce).

Retention varies by language (69.6%–96.4% for this model's 10 languages — see table above). Yoruba and Igbo's lower retention is primarily genuine ASR-CER disagreement rather than bad data — both are tonal/diacritic-heavy languages where omniASR_CTC_1B performs less reliably.

Training

• Base: VoxCPM2_langtagged — openbmb/VoxCPM2 with 10 <lang:XX> tokens added and the text embedding table resized (mean-initialized rows) before any fine-tuning. Same tagged base reused for the Top-5 sibling model.
• Mode: full fine-tuning (all parameters trainable except the frozen AudioVAE codec).
• Data balancing: sqrt-temperature resampling (weight ∝ √hours, renormalized to preserve total volume) across the 10 languages, same methodology as the other models in this collection.
• Scale: 2,237,047 training rows (clips) after resampling, 4,297 held-out validation rows.
• Steps: 34,000 steps (~1.01 epochs over the resampled training set), batch size 3 × grad-accum 11 × 2 GPUs (effective batch 66).
• Hardware: 2× NVIDIA H200, via torchrun DDP.
• Learning rate: 1e-5, 100 warmup steps.
• Final validation loss: total 0.851 (diff 0.835, stop 0.016) at step 33,999.

Usage

from huggingface_hub import snapshot_download
from voxcpm import VoxCPM
import soundfile as sf

model_dir = snapshot_download("waxal-benchmarking/voxcpm2-top10-full")
model = VoxCPM.from_pretrained(model_dir, load_denoiser=False, optimize=False)

wav = model.generate(
    text="<lang:om> Akkam bultan?",
    cfg_value=2.0,
    inference_timesteps=22,
)
sf.write("out.wav", wav, model.tts_model.sample_rate)

optimize=False is required — torch.compile (the default optimization path) conflicts with some inference environments (e.g. HF ZeroGPU Spaces); disabling it costs some speed but avoids crashes.

Voice cloning works the same way as base VoxCPM2 — pass prompt_wav_path and prompt_text (the literal transcript of what's spoken in the reference clip) alongside the tagged text.

Evaluation

Pending. The automated evaluation pipeline used for earlier releases (ASR-based CER against ground-truth text, a real-recording CER noise floor per language, and DNSMOS naturalness) has not yet been run against this checkpoint. Numbers from voxcpm2-multilang28-full are not applicable here — that model was trained on uncleaned data and is not a valid proxy for this model's quality. This section will be updated once Stage A/B/C evaluation has run.

Known Limitations

• No evaluation results yet (see above) — training loss converging normally is not the same as verified downstream speech quality.
• Yoruba and Igbo retained less data than most of the other languages (74.0% and 69.6%) due to the ASR-CER gate disagreeing more often on these tonal/diacritic-heavy languages. We did not disable the CER gate for these two (unlike Somali/Ikposo in the all-28 cleaning pass, where drop rates were far more extreme at 70-80%) — their elevated-but-not-extreme drop rates were judged plausible rather than clearly an ASR-model artifact, but this is a judgment call, not a certainty.
• Voice-design and cross-lingual flexibility are likely reduced, consistent with what we've seen in other full-fine-tune checkpoints in this collection — full-FT on read-speech-only data tends to degrade the base model's text-prompted style/voice-design control and non-trained-language generation.
• Non-terminating generation was an observed failure mode in earlier releases on this same base model (see voxcpm2-multilang28-full's model card); whether it recurs here is one of the things the pending evaluation will check.

Related

• waxal-benchmarking/voxcpm2-top5-full — the 5-language sibling model, same base checkpoint and methodology, a subset of this model's languages.
• waxal-benchmarking/voxcpm2-multilang28-full and voxcpm2-multilang10-full — earlier releases trained on the uncleaned pooled corpus.
• All-28 (clean-data) sibling model — trained the same way as this one, for comparison; linked here once released.
• AfroNet speech collection — the source training data.