jsbeaudry/Fatiman-TTS-v2-onnx

Fatiman-TTS-v2 — ONNX for phones

Fatiman-TTS-v2, the Haitian Creole fine-tune of Pocket TTS trained on human speech only, exported to ONNX for on-device use with ONNX Runtime. The files and the interface are the same as Fatiman-TTS-v1-onnx, so an app that loads v1 can load v2 by changing the repository name.

Files

file	what	size
text_conditioner_int8.onnx	token ids → text embeddings	16 MB
flow_lm_main_kv_q4.onnx	24-layer backbone LM, append-only KV, 4-bit weights (MatMulNBits, block 32)	190 MB
flow_lm_flow_int8.onnx	flow head: conditioning + noise → 32-dim latent (one LSD step)	10 MB
mimi_decoder_int8.onnx	Mimi decoder: latents → 24 kHz audio, streaming state (identical to v1's)	23 MB
bundle.json	dims, state manifests, kv layout, lm_file, Mimi pass-through outputs (same as v1)
tokenizer.json	the fine-tune's Unigram tokenizer
bos_before_voice.f32	BOS embedding inserted before the voice prompt (float32, 1024)
voices/*.f32, voices/voices.json	the nine voice prompt embeddings (float32 [1, frames, 1024]), computed in PyTorch for v2

The LM was exported with the scripts in Fatiman-TTS-v1-onnx/export. There is no INT8 alternative LM for v2 yet.

The LM interface (append-only KV)

• inputs: sequence [1, S, 32] (NaN = BOS), text_embeddings [1, X, 1024], past_k_i / past_v_i [1, P, 16, 64] for i = 0..23 (P may be 0)
• outputs: conditioning [1, 1024], eos_logit [1, 1], new_k_i / new_v_i [1, S+X, 16, 64]

1. Voice: run once with text_embeddings = [bos_before_voice, voice frames] and empty past. Keep the outputs as that voice's prefix.
2. Each text chunk: reset the cache to the voice prefix, then run the text embeddings.
3. Each frame: run sequence = the previous latent (NaN for the first frame) → flow head (s=0, t=1, x ~ N(0, 0.7), x += flow) → latent.
4. Stop a few frames after eos_logit > -4.

Spell numbers and symbols out before speaking (17 → disèt), as in training.

Quality

A local check of this export against v1's (3 voices × 8 sentences) gave the same average CER, about 4.9%, for both. For the full results of the model, see the Fatiman-TTS-v2 card.

Consent

The nine voices are published with their speakers' signed consent. Never use this model to imitate someone's voice without their permission, or to deceive.

License

CC-BY-4.0, as the base model. Based on Pocket TTS by Kyutai.