EMTIAZZ/xtts-bangla-checkpoints

XTTS v2 Bangla — Training Checkpoints

Zero-shot voice cloning — provide just 5 seconds of any speaker's audio and the model will synthesize Bangla speech in that voice. No retraining required.

This repository contains 5 training checkpoints from the XTTS v2 Bangla fine-tuning run, evenly spaced across steps 382,000–406,000.

For the production-ready best model (step 406,521), see: 👉 EMTIAZZ/xtts-bangla-finetuned

---

Checkpoints

File	Step	Notes
checkpoint_382000.pth	382,000	Earliest checkpoint in this set
checkpoint_389000.pth	389,000
checkpoint_395000.pth	395,000
checkpoint_401000.pth	401,000
checkpoint_406000.pth	406,000	Latest regular checkpoint

These are full model checkpoints ready for inference (GPT + HiFi-GAN + mel stats). Use checkpoint_406000.pth for best results, or compare earlier steps to study the training progression.

---

Quick Start

1. Install dependencies

pip install "TTS==0.22.0" "transformers==4.36.0" "tokenizers==0.15.0" soundfile

Important: Use these exact versions. Newer transformers versions break XTTS inference.

2. Run inference

python inference.py --text "আমি বাংলায় কথা বলতে পারি।" --checkpoint checkpoint_406000.pth --output output.wav

Clone any voice with just 5 seconds of audio:

python inference.py \
    --text "আমি বাংলায় কথা বলতে পারি।" \
    --checkpoint checkpoint_406000.pth \
    --ref_audio your_speaker.wav \
    --output output.wav

The --ref_audio file can be as short as 5 seconds. The model extracts a speaker embedding from it and generates speech in that voice. WAV or MP3 works.

---

Python API

import soundfile as sf
from huggingface_hub import snapshot_download
from TTS.tts.configs.xtts_config import XttsConfig
from TTS.tts.models.xtts import Xtts

# Download all files (~26 GB total)
model_dir = snapshot_download("EMTIAZZ/xtts-bangla-checkpoints")

# Choose a checkpoint — use checkpoint_406000.pth for best results
checkpoint_path = f"{model_dir}/checkpoint_406000.pth"

# Load model
config = XttsConfig()
config.load_json(f"{model_dir}/config.json")
model = Xtts.init_from_config(config)
model.load_checkpoint(
    config,
    checkpoint_dir=model_dir,
    checkpoint_path=checkpoint_path,
    vocab_path=f"{model_dir}/vocab.json",
    use_deepspeed=False,
    eval=True,
)
model.cpu()   # or model.cuda() on GPU

# Speaker conditioning — just 5 seconds of audio is enough
gpt_cond_latent, speaker_embedding = model.get_conditioning_latents(
    audio_path=f"{model_dir}/reference_audio.wav",
    gpt_cond_len=config.gpt_cond_len,
    max_ref_length=config.max_ref_len,
    sound_norm_refs=config.sound_norm_refs,
)

# Generate speech — use language="en" (NOT "bn")
outputs = model.inference(
    text="আমি বাংলায় কথা বলতে পারি।",
    language="en",
    gpt_cond_latent=gpt_cond_latent,
    speaker_embedding=speaker_embedding,
    temperature=0.1,
    length_penalty=1.0,
    repetition_penalty=10.0,
    top_k=10,
    top_p=0.3,
)
sf.write("output.wav", outputs["wav"], 24000)

---

Important: Why language="en"?

This model was fine-tuned using language="en" to bypass XTTS's language-specific tokenizer preprocessing. The Bangla text is encoded directly as Unicode characters.

Using language="bn" will produce garbled/wrong audio. Always use language="en" when running inference, even for pure Bangla text.

---

Inference Parameters (Production)

Parameter	Value	Notes
temperature	0.1	Low = more deterministic
length_penalty	1.0	Default
repetition_penalty	10.0	Prevents repetition/loops
top_k	10
top_p	0.3
language	"en"	Required

---

Files

File	Size	Description
checkpoint_382000.pth	5.3 GB	Step 382,000
checkpoint_389000.pth	5.3 GB	Step 389,000
checkpoint_395000.pth	5.3 GB	Step 395,000
checkpoint_401000.pth	5.3 GB	Step 401,000
checkpoint_406000.pth	5.3 GB	Step 406,000
config.json	5 KB	Model configuration
vocab.json	331 KB	BPE vocabulary (8,565 Bangla tokens)
reference_audio.wav	295 KB	Sample reference speaker audio (Adiba voice)
inference.py	—	Ready-to-run inference script

---

Training Details

Base model	Coqui XTTS v2 (tts_models/multilingual/multi-dataset/xtts_v2)
Dataset	~99K Bangla speech samples
Training platform	Modal (H100 GPU)
Fine-tuning date	January 2026
Vocabulary	8,565 BPE tokens (Bangla-extended)
Output sample rate	24,000 Hz

---

Troubleshooting

ImportError from transformers / tokenizers mismatch Install exact versions: pip install "transformers==4.36.0" "tokenizers==0.15.0"

Audio is garbled or very short Make sure you are using language="en", not "bn".

Slow inference on CPU XTTS has known MPS issues on Apple Silicon. CPU is the safest option. On Linux/Windows with NVIDIA GPU, use model.cuda() for fast inference.

Out of memory Each checkpoint is ~5.3 GB. On CPU you need ~6 GB RAM. On GPU you need ~6 GB VRAM.