rajanishpoudel/vits-ne-multispeaker

VITS Nepali Multi-Speaker

A VITS text-to-speech model for Nepali (ने, Devanagari script), trained with Coqui TTS on 3 speakers. Grapheme-based (no phonemizer required), 22,050 Hz output. Data the initial model is trained on can be found here https://huggingface.co/datasets/davidguzmanr/open-bible-resources.

Speakers

ID	Training utterances
SPEAKER_00	12,579
SPEAKER_01	7,548
SPEAKER_02	6,465

Samples

samples/ has a few quick previews synthesized from this checkpoint:

• sample_SPEAKER_00_short.wav — short sentence, SPEAKER_00
• sample_SPEAKER_0{0,1,2}_long.wav — same 28-word sentence across all 3 speakers

Usage

pip install coqui-tts huggingface_hub

from huggingface_hub import hf_hub_download
from TTS.utils.synthesizer import Synthesizer

repo_id = "rajanishpoudel/vits-ne-multispeaker"
model_path = hf_hub_download(repo_id, "best_model.pth")
config_path = hf_hub_download(repo_id, "config.json")
speakers_path = hf_hub_download(repo_id, "speakers.pth")

synth = Synthesizer(
    tts_checkpoint=model_path,
    tts_config_path=config_path,
    tts_speakers_file=speakers_path,
    use_cuda=False,  # True if you have a GPU
)

wav = synth.tts(text="नमस्ते, तपाईंलाई कस्तो छ?", speaker_name="SPEAKER_00")
synth.save_wav(wav, "out.wav")

Speaker names are SPEAKER_00, SPEAKER_01, SPEAKER_02.

Training

• Architecture: VITS, multi-speaker (speaker embedding, no d-vectors)
• Text representation: graphemes, basic_cleaners, 82-character vocabulary
• Checkpoint: step 66,360 — selected as the run's best on aggregate eval loss; training continued well past this point without the aggregate metric finding a new best, so this remains the published checkpoint. See the training run's notes for the fuller picture (aggregate loss plateaued, but mel-reconstruction loss kept drifting down slowly).
• Data: Bible-verse recordings, Nepali config, 3 speakers, ~26.6k utterances total (train+test) after cleaning.

License / data provenance

Please verify licensing before relying on this model for anything beyond personal or research use. The training audio comes from a Bible-recordings dataset aggregated on the Hub; its own license terms were not stated on the source dataset at the time this model was trained, and audio Bible recordings distributed by e.g. Faith Comes By Hearing are commonly limited to non-commercial use even where the underlying text is public domain. This model card ships with license: other as a placeholder — replace it once you've confirmed the actual terms that apply to derivative (voice-model) use of the source recordings.

Limitations

• One speaker (SPEAKER_00) is noticeably slower-paced than the other two on identical text in informal listening checks — may be a duration-predictor quirk rather than the speaker's natural pace; not fully diagnosed.
• Trained on read Bible-verse speech only — expect degraded quality on conversational, colloquial, or code-switched Nepali text.