AhsanTalal/urdu-matcha-tts

Matcha Urdu TTS

A fast, single-speaker Urdu text-to-speech model based on Matcha-TTS (conditional flow matching, ICASSP 2024)

• Output: 80-band mel spectrogram → waveform
• Text input: Urdu graphemes
• Speakers: 1

Audio examples

Listen to these samples (generated by this model):

Example 1

کیا آپ جانتے ہیں کہ میں ایک اردو ٹیکسٹ ٹو سپِیچ ہوں؟

Example 2

سوچتا کوئی اور ہے، ڈھلتا ہے لفظ میں، میں فقط ایک آئینہ ہوں، میرا نام ہے رقم۔

Example 3

اردو ٹیکسٹ ٹو سپِیچ کو استعمال کرنے کے لیے نیچے دیے ہوئے کوڈ کا استعمال کریں۔

File	Description
matcha_tts.onnx	Acoustic model in ONNX format (10 steps, opset 17)
matcha_urdu.ckpt	PyTorch / Lightning checkpoint (same model)
vocab.json	Character vocabulary (char → id)
symbols.txt	Same vocabulary, one symbol per line
config.yaml	Model configuration reference

Install the model

pip install --upgrade huggingface_hub
huggingface-cli download AhsanTalal/urdu-matcha-tts matcha_tts.onnx vocab.json matcha_urdu.ckpt symbols.txt config.yaml --local-dir .

Install the runtime + a Vocos vocoder:

pip install onnxruntime numpy soundfile torch torchaudio vocos

The model outputs mel spectrograms — Vocos converts them to audio. Vocos itself is a separate universal model (22.05 kHz), loaded from its own repo below.

How to use (ONNX)

import json
import numpy as np
import torch
import onnxruntime as ort
from vocos import Vocos

# 1. Encode Urdu text (intersperse blank = 0)
stoi = json.load(open("vocab.json", encoding="utf-8"))
space_id = stoi[" "]
text = "ہماری ویب سائٹ پر خوش آمدید"
ids = [stoi.get(ch, space_id) for ch in text]
inter = [0] * (len(ids) * 2 + 1)
for i, v in enumerate(ids):
    inter[i * 2 + 1] = v
x = np.array([inter], dtype=np.int64)

# 2. Run the acoustic model -> 80-band mel
sess = ort.InferenceSession("matcha_tts.onnx", providers=["CPUExecutionProvider"])
scales = np.array([0.667, 1.0], dtype=np.float32)   # [temperature, length_scale]
mel, mel_lengths = sess.run(None, {
    "x": x,
    "x_lengths": np.array([len(inter)], dtype=np.int64),
    "scales": scales,
})

# 3. Install & run Vocos to get audio
vocos = Vocos.from_pretrained("charactr/vocos-mel-22khz")
audio = vocos.decode(torch.from_numpy(mel))

import soundfile as sf
sf.write("out.wav", audio.numpy(), 22050)

How to use (PyTorch checkpoint)

You can also run the original PyTorch checkpoint instead of the ONNX — useful if you want to fine-tune or use the training code. Install from Hugging Face:

from huggingface_hub import hf_hub_download
import torch
from matcha.models.matcha_tts import MatchaTTS

# 1. Download the checkpoint
ckpt = hf_hub_download("AhsanTalal/urdu-matcha-tts", "matcha_urdu.ckpt")
symbols = hf_hub_download("AhsanTalal/urdu-matcha-tts", "symbols.txt")

# 2. Load it (see Matcha-TTS repo: pip install -e Matcha-TTS)
model = MatchaTTS.load_from_checkpoint(ckpt, map_location="cpu", weights_only=False)
ds = torch.load(ckpt, map_location="cpu", weights_only=False)["hyper_parameters"]["data_statistics"]
model.mel_mean = torch.tensor(ds["mel_mean"])   # load_from_checkpoint does NOT restore these
model.mel_std = torch.tensor(ds["mel_std"])
model.eval()

# 3. Synthesize -> mel, then vocode with Vocos as above
out = model.synthesise(x, x_lengths, n_timesteps=10, temperature=0.75, length_scale=0.8)

Fine-tune it

The checkpoint is a standard PyTorch Lightning checkpoint, so you can fine-tune it with the original Matcha-TTS training code on your own Urdu dataset:

git clone https://github.com/shivammehta25/Matcha-TTS
cd Matcha-TTS
pip install -e .
# replace matcha/text/symbols.py with symbols.txt (350 symbols, keep graphemes)
python train.py configs/matcha/urdu.yaml   # or your own config, resume from matcha_urdu.ckpt

Keep text_cleaner = "urdu_cleaners_graphemes" (grapheme-based, no phonemizer) and the same 350-symbol vocab when fine-tuning.

Training data

Trained on UrduSpeech, a public Urdu speech dataset on Hugging Face.

Model details

• Architecture: Matcha-TTS (non-autoregressive, optimal-transport conditional flow matching decoder, euler solver, 10 steps)
• Text processing: grapheme-based Urdu, blank token interspersed between every character (t0 [blank] t1 [blank] ... [blank] tN [blank])
• Acoustic features: 80-band mel, n_fft=1024, hop_length=256, win_length=1024, 22.05 kHz, f_max=8000
• Normalisation: mel mean = -1.1573, std = 2.2750 (from hyper_parameters.data_statistics)

Limitations

• Single speaker, single style; not designed for voice cloning or code-switching.
• Text outside the vocabulary is mapped to spaces.

References

• Matcha-TTS paper: Mehta et al., "Matcha-TTS: A fast TTS architecture with conditional flow matching", ICASSP 2024 — https://arxiv.org/abs/2309.03199
• Vocos: Lee et al., "Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis", ICLR 2024