Saunak359/IndicF5-Multilingual-Conversational

IndicF5 Multilingual Conversational

Fine-tuned IndicF5 model for Hindi, Bengali and Punjabi conversational speech synthesis.

Base model

ai4bharat/IndicF5

Training dataset

Saunak359/Multi-Lingual-Curated_Dataset

Final split:

Split	Samples
Train	352
Validation	59
Test	29

All training audio was mono 24 kHz.

Tokenizer OOV rate across train, validation and test was 0%.

Pretrained IndicF5 parameter loading coverage was 100%.

Training

Training was performed on an NVIDIA T4 using full FP32 because FP16 produced non-finite F5 flow-matching losses on otherwise valid inputs.

Stage 1

• LR: 1e-5
• Epochs: 2
• Best validation loss: 0.595476

Stage 2

• LR: 5e-6
• Epochs: 2
• Best validation loss: 0.594493

Stage 3

• LR: 2e-6
• Epochs: 1
• Best validation loss: 0.594424

The online model outperformed EMA during checkpoint selection.

Final evaluation

Final fixed validation loss:

0.594424

Held-out test loss:

0.643411

The test split was evaluated only after final checkpoint selection and was not used for further tuning.

Per-language test results are available in:

results/final_metrics.json

Because the held-out test split contains only 29 samples, per-language F5 loss values should be interpreted cautiously.

Generated evaluation samples

The repository includes held-out synthesis examples under:

generated_samples/

for Hindi, Bengali and Punjabi.

Their corresponding target/reference information is stored in:

results/generation_manifest.csv

Model architecture

• CFM + DiT
• DiT dimension: 1024
• Depth: 22
• Attention heads: 16
• Text dimension: 512
• Mel channels: 100
• Sample rate: 24 kHz
• Vocoder: Vocos

Inference

Install the AI4Bharat IndicF5/F5-TTS implementation and see:

inference_example.py

Inference requires:

1. reference audio
2. exact reference transcript
3. target text

Files

Final model

model.safetensors

Training checkpoints

• checkpoints/stage1_foundation_best.safetensors
• checkpoints/stage2_continuation_best.safetensors
• checkpoints/stage3_final_best.safetensors
• checkpoints/vocab.txt

Results

• results/final_metrics.json
• results/stage1_history.csv
• results/stage2_history.csv
• results/stage3_history.csv
• results/generation_manifest.csv

Reproducibility

• training_config.json
• manifests/train_manifest.csv
• manifests/validation_manifest.csv
• manifests/test_manifest.csv

Intended use

Research and development of multilingual Indian conversational TTS and voice-agent systems.

Only use reference voices for which appropriate consent and usage rights are available.