Swamiji VoxCPM2: multilingual-vox-good (English, Telugu, Kannada; LocDiT frozen)

Swamiji's voice in English, Telugu and Kannada from one VoxCPM2 model. It needs no reference clip, because the voice is in the weights. Research model. Its stage 3 trained with LocDiT (feat_decoder, 212 M parameters) frozen, so the acoustic decoder is still the one from the English stage 2. (Named best-multilingual-vox on 2026-10-30, best-multilingual on 2026-10-29; first named voxcpm2-indic-multi-stage3-te-kn-themes-frozen-locdit-lr1e-4-5ep. The same recipe with nothing frozen is sw-voice/voxcpm2-indic-multi-stage3-navaratri-te-kn-themes-lr1e-4-5ep, which held this name on 2026-10-28. On 2026-10-31 the best-* names moved to the newer picks (sw-voice/best-2-multilingual-maya, sw-voice/best-2-multilingual-vox) and this repo went back to its first name, then was named multilingual-vox-good (2026-10-31); the old names still redirect here.)

How it was trained

Three stages with the official VoxCPM trainer. The AudioVAE was never trained.

Stage	Starts from	Data	Recipe
1. Multilingual base	openbmb/VoxCPM2	7 Indian languages: IndicTTS + Rasa, te kn ta ml sa mr gu; 167k clips, 453 h of training passes; no Swamiji	LR 5e-5, warmup 15%, 1 pass, batch 1 × 16, 14,767 steps (sw-voice/voxcpm2-indic-multi-lr5e-5-1ep)
2. English Swamiji	stage 1	sw-voice/swamiji-voice-sft-sidon: 958 English clips, 5.1 h, Sidon-restored	LR 1e-4 cosine, warmup 15%, 10 epochs, batch 1 × 16, 600 steps, all parameters (sw-voice/voxcpm2-indic-multi-stage2-english-sidon-lr1e-4-10ep)
3. Telugu + Kannada (this model)	stage 2	swamiji-navaratri-telugu-sidon-themes + swamiji-navaratri-kannada-sidon-themes: 127 + 124 whole-theme units (up to ~90 s, never mid-sentence, real pauses kept), 3.7 h, Sidon-restored	LR 1e-4 cosine, warmup 15% (48 steps), 5 epochs, batch 1 × 4, 315 steps; feat_decoder.* frozen (kept out of AdamW), every other parameter trained; one H100

How it scores

Ten standard answers per language, generated reference-free, sentence by sentence, cfg 2, 10 steps, seed 0.

	English	Telugu	Kannada
Voice match to real Swamiji (WavLM-ECAPA, % of real-vs-real)	83.0%	96.4%*	98.3%*
Word errors (EN: Whisper large-v3 WER; others: Sarvam WER / CER)	1.5%	8.3% / 2.2%	7.5% / 2.4%
Speaking rate, % of Swamiji's	132%	108%	114%
End silence (mean)	0.23 s	0.30 s	0.26 s
Voice clarity (CPPS, dB; his English recordings 10.3, his Navaratri recordings 8.7–8.9)	8.9	8.7	9.3

* The real Telugu and Kannada clips come from the same Navaratri recordings the model trained on, so these read high.

For comparison, the unfrozen stage 3 (voxcpm2-indic-multi-stage3-navaratri-te-kn-themes-lr1e-4-5ep) scores 91.8 / 102.0 / 103.2% voice match and CER 1.9% / 2.6%.

Known traits

• Telugu and Kannada are close to his voice and read accurately (CER 2.2% / 2.4%).
• English is the weakest of our multilingual models (83.0% voice match) and speaks about a third faster than Swamiji.
• Slightly rough timbre. Freezing LocDiT did not remove the roughness of the Navaratri recordings (CPPS 8.7–9.3); it comes through the language model's conditioning, not the decoder.
• Serve it sentence by sentence. Whole multi-sentence answers can skip lines on the Telugu-trained VoxCPM2 models.

Usage

from huggingface_hub import snapshot_download
from voxcpm import VoxCPM

path = snapshot_download("sw-voice/multilingual-vox-good")
model = VoxCPM(voxcpm_model_path=path, zipenhancer_model_path=None, enable_denoiser=False)
wav = model.generate(text="మనస్సు ప్రశాంతంగా ఉండాలంటే ప్రతిరోజూ ధ్యానం చేయాలి.", cfg_value=2.0,
                     inference_timesteps=10, seed=0)
# 48 kHz mono; no reference audio or prompt text needed

Code: swamiji-tts research/multi-stages (run_stage3frz.sh, VARIANT=A). Samples: the Space sw-voice/swamiji-telugu-kannada-samples. train_plan.json holds the stage-3 schedule.