indic-speak-mlx-6bit

Built with Indic-Speak from Bodhan AI / AI4Bharat.

This is a community MLX conversion of bodhan-ai/indic-speak, a Llama-3.2-3B based text-to-speech model for 22 Indian languages and English. It is not an official Bodhan AI release. All credit for the model goes to Bodhan AI / AI4Bharat, IIT Madras.

Precision	6-bit (group size 64), 6.5 bits/weight
LM weights	2.5 GB
Decode speed on M3 Max (36 GB)	~47 tok/s

Slower than 4-bit in mlx-lm 0.31 (no fast 6-bit kernels). Kept for quality comparisons.

Other variants: 4bit · 6bit · 8bit · mixed · bf16

What's in the repo

• MLX weights + tokenizer for the speech LM (converted with mlx_lm.convert)
• vocos/ — Bodhan's fine-tuned Vocos decoder (unchanged from upstream)
• inference.py — upstream prompt/SNAC helpers (unchanged)
• tts_mlx.py — MLX synthesizer with streaming and sentence chunking (this conversion's addition)
• voices.md, token_contract.md, and the license files from upstream

The SNAC quantizer (hubertsiuzdak/snac_24khz) is downloaded on first use.

Usage

pip install mlx-lm torch snac soundfile huggingface_hub numpy

from huggingface_hub import snapshot_download
import sys; sys.path.insert(0, snapshot_download("iniquitous/indic-speak-mlx-6bit"))
from tts_mlx import MLXTTS, speak_long, stream_long
import soundfile as sf

tts = MLXTTS("iniquitous/indic-speak-mlx-6bit")       # loads LM (MLX) + SNAC + Vocos (torch/MPS)

# single sentence
wav, stats = tts("नमस्ते, आज हम विज्ञान पढ़ेंगे।", speaker="Amit", style="HAPPY", seed=1234)
sf.write("out.wav", wav, tts.sample_rate)
print(stats)   # tokens, tok_per_s, ttft, rtf ...

# multi-sentence text (recommended for quantized variants — see caveats)
wav, per_sentence = speak_long(tts, long_text, speaker="Kavya")

# streaming: yields float32 chunks (~0.7 s) while generation is still running
for chunk, st in stream_long(tts, long_text, speaker="Kavya"):
    play(chunk)   # your audio sink

Voices: one female and one male per language (voices.md), e.g. Hindi Kavya/Amit, Tamil Anitha/Arun. Any voice can speak any language. Style tags trained upstream are uppercase emotion labels (ANGER, HAPPY, SAD, FEAR, …) and free-text phrases; lowercase delivery tags such as educational lecture are less reliable.

Important: RoPE config patch

transformers v5 writes the Llama-3 RoPE settings under rope_parameters. mlx-lm (≤ 0.31) reads rope_theta and rope_scaling instead, so a naive mlx_lm.convert of the upstream repo silently runs with rope_theta=10000 and no llama3 scaling. The result still produces audio, but generation is unstable (random early stops, runaway silence, 2–4× duration variance between seeds). The config.json here carries both key sets, so it loads correctly. If you convert upstream yourself, copy rope_parameters into rope_theta + rope_scaling.

Caveats

• Long inputs: quantized variants tend to stop early or drift on multi-sentence prompts. speak_long / stream_long split on sentence boundaries (। . ! ?), cap tokens per sentence, and trim silence. Use them.
• Repetition penalty: upstream's production config uses 1.2; tts_mlx.py defaults to it.
• Streaming seams: Vocos is non-causal, so stream() holds back 4 frames of lookahead. The streamed waveform matches the offline decode to < 1e-4.
• Speed numbers above are single-stream on an M3 Max and drop ~25 % when the chip is thermally throttled.
• Quality was checked on Hindi by duration/silence statistics and listening, not by a formal WER/MOS benchmark.

License

The model weights are released under the Indic Open Model License v1.0 (Bodhan AI / AI4Bharat), the same license as the upstream model, included verbatim as Bodhan_AI_Open_Model_License.md (full text) and indic-open-license.md (plain-language deed). Key points: attribution is required, derivatives carry the same license, hosting the model as a service for third parties needs Bodhan AI's written approval, and the prohibited-uses list applies. The upstream model is itself built on Llama 3.2 and subject to the Llama 3.2 Community License.

tts_mlx.py (the conversion's own code) is released under the same terms.