tryorato/orato-tts-hindi-v1

Orato TTS — Hindi v1

Fine-tune of ai4bharat/IndicF5 (F5-TTS Base (DiT) — character tokenizer, Devanagari vocab) on ~194 hours of Hindi/Hinglish calling-domain speech (support, delivery, payments, recharge, appointments). Multi-speaker via reference audio — one checkpoint, switch male/female by choosing a voice pack entry.

What's in this repo

File	Purpose
model.pt	Weights-only F5-TTS checkpoint (~1.3 GB; no optimizer state)
vocab.txt	IndicF5 Devanagari character vocab — must match these weights
voices/male.wav, voices/female.wav	Frozen reference clips for live demos
voices.json	Exact transcript for each reference wav

Voice	File	Notes
male	voices/male.wav	Reference transcript stored in voices.json
female	voices/female.wav	Reference transcript stored in voices.json

Training

• Base: IndicF5 (F5-TTS Base / DiT), float32 (bf16 NaN'd mid-run)
• Data: ~194 h Hindi/Hinglish (mixed real + synth calling-domain)
• Epochs: 3 · LR 1e-5 · batch 19200 frames/GPU · 1× H100
• Tokenizer: character · vocab size 2545 (IndicF5)
• Final update: 10659 · checkpoint: model_last.pt → exported weights-only

Evaluation (base IndicF5 vs Orato TTS)

Scored on the same held-out Hindi prompts and the same male/female references. ASR-CER uses Whisper large-v3 (Hindi); speaker SIM uses SpeechBrain ECAPA. CER on number/loanword lines is noisy (Whisper often writes OTP 5732 / 399 while the prompt is Devanagari) — treat as directional; listening is primary.

Voice	Base CER	Fine-tuned CER	Base SIM	Fine-tuned SIM
female	19.24%	18.34%	0.863	0.899
male	18.93%	18.24%	0.900	0.903

Fine-tuning gives a small CER gain and a clearer female speaker-SIM lift (0.86 → 0.90). Male SIM was already strong on the base model.

Switching speakers at inference

There is no separate male/female model. Pass a different reference:

say("नमस्ते, ओराटो की ओर से आपका स्वागत है।", voice="female")
say("नमस्ते, ओराटो की ओर से आपका स्वागत है।", voice="male")

Use gender-neutral Hindi in shared demos, or swap verb endings (रहा हूँ / रही हूँ) to match the chosen voice. Prefer Devanagari for loanwords (ओटीपी, ईएमआई, कस्टमर केयर) — Latin English is the weaker path.

Loading

import json, torch
from pathlib import Path
from huggingface_hub import snapshot_download

snap = Path(snapshot_download("tryorato/orato-tts-hindi-v1"))
voices = json.loads((snap / "voices.json").read_text(encoding="utf-8"))
voice = "female"   # or "male"
ref_wav  = snap / voices[voice]["wav"]
ref_text = voices[voice]["ref_text"]
ckpt     = snap / "model.pt"
vocab    = snap / "vocab.txt"
# Load with F5-TTS / your Orato inference wrapper — MUST use this vocab.txt
# Prefer Devanagari prompts (ओटीपी, कस्टमर केयर) over Latin English.

Must use this repo's vocab.txt. Loading these weights with F5-TTS's default Chinese pinyin vocab produces gibberish (every Hindi character maps to filler).

Intended use / limitations

• Beta. Tuned for Hindi calling-domain prompts; not evaluated as a general multilingual TTS.
• Reference-conditioned cloning only — no fixed speaker IDs without a ref wav.
• Do not clone voices without explicit permission (IndicF5 Terms of Use).
• Access is currently private.

Attribution & data rights

• Base model: AI4Bharat IndicF5 (MIT; gated — accept their terms before redistributing derivatives publicly).
• Architecture: F5-TTS (SWivid et al.).
• Training data: mix of public research sources (e.g. Rasa Hindi, IndicTTS) under their published licenses, plus proprietary Orato calling-domain audio.
• This repository publishes weights + reference pack only — no raw training audio or transcripts.
• Commercial-use / consent review for proprietary data is in progress. Released for internal/beta evaluation until that review completes.