yogenghodke/indic-f5-mlx

IndicF5-MLX

Native MLX inference for AI4Bharat's IndicF5 — zero-shot, multilingual Indian-language text-to-speech running directly on Apple Silicon (no PyTorch/MPS).

IndicF5 is a fine-tune of F5-TTS covering 11 Indian languages (Hindi, Marathi, Tamil, Telugu, Kannada, Bengali, Gujarati, Malayalam, Odia, Punjabi, Assamese). This repo makes it run in MLX by adapting lucasnewman/f5-tts-mlx.

Why this isn't a trivial re-use of f5-tts-mlx

IndicF5 is F5-TTS v0 (F5TTS_Base); f5-tts-mlx ports F5-TTS v1. The two are shape-identical, so IndicF5's weights load cleanly into the v1 model — but the audio comes out as speech-like, wrong-language gibberish. The cause is a single behavioral difference:

RoPE application order. F5 v0 (IndicF5) applies the rotary position embedding to q/k before splitting them into attention heads; f5-tts-mlx (v1) applies it after. Same weights, different attention geometry → the acoustic model is fine but the text conditioning is scrambled, so the model ignores the text.

This was isolated by numerically diffing every stage against the PyTorch reference:

stage	max-abs diff (v1 order)	after fix
text_embed	0.00001 ✓	—
input_embed (grouped conv)	0.00002 ✓	—
time_embed	0.0000 ✓	—
attn_norm (AdaLN)	0.0000 ✓	—
attention output	38.4 ✗	0.008 ✓

indic_f5_mlx restores the v0 order with a small runtime patch (patch_f5_rope_to_v0()), plus the other IndicF5-specific settings (text_mask_padding=False, text_num_embeds=len(vocab), raw-char tokenization, and skipping the checkpoint's bundled vocoder).

Install

pip install git+https://github.com/yogen-ghodke-113/indic-f5-mlx

Requires an Apple-Silicon Mac. Depends on mlx, f5-tts-mlx, vocos-mlx, soundfile, huggingface_hub.

Access to the weights

IndicF5 is MIT-licensed but gated on the Hub. Accept the terms on the model page and log in once:

huggingface-cli login

This library downloads and converts the gated weights at load time — it does not re-host them.

Usage

from indic_f5_mlx import load_indicf5, generate
import soundfile as sf

model, _ = load_indicf5()   # downloads + converts ai4bharat/IndicF5 to MLX

audio = generate(
    model,
    ref_audio_path="reference.wav",   # a clean 24 kHz mono clip, ~5–10 s
    ref_text="transcript of the reference clip",
    text="भारतीय संविधानाच्या अनुच्छेद तीनशे सत्तर नुसार ...",
)
sf.write("out.wav", audio, 24000)

Notes that matter for good output

• Reference audio must be 24 kHz mono. Convert with ffmpeg -i in.wav -ac 1 -ar 24000 -sample_fmt s16 -t 10 ref.wav. A clean natural voice clip works best.
• Spell numbers out as words (तीनशे सत्तर, not 370) — F5 reads digit strings unreliably.
• Long text is split per sentence internally and concatenated.
• Speed: F5 is non-autoregressive but runs steps × 2 (CFG) full-sequence passes. On a base M4 (~10 GPU cores) expect ~0.2–0.7× realtime depending on steps; it's much faster on an M-series Max.

Attribution & license

This project is MIT-licensed. It stands on:

• AI4Bharat / IndicF5 — the model (MIT)
• SWivid / F5-TTS — the architecture (MIT)
• lucasnewman / f5-tts-mlx — the MLX implementation this adapts (MIT)
• ml-explore / MLX and vocos-mlx

Please cite IndicF5 and F5-TTS if you use this. Use voice cloning responsibly and only with consent for the reference voice.