travis-moore/sema-tts-boost

Sema TTS Boost v1

Sema TTS Boost is a targeted continuation of sema-tts-afro: the same 24-variety African text-to-speech model with zero-shot voice cloning, further trained to lift its weakest languages — Hausa, Yoruba, Amharic, Twi, West African Pidgin and English (including African accents) — while a replay mixture holds the other languages in place.

Built under Sema by Angelo Asante.

Hear it

Every clip below is one voice — the author's — cloned from a single 9-second reference, speaking languages the reference never spoke:

Language	Sample
Twi
English
Pidgin
Hausa
Yoruba
Swahili

What changed vs the base

• ~313 hours of additional training audio: 213 hours of fresh speech concentrated on the six focus languages (including studio-quality recordings and multi-accent English covering US and Southern African speakers), plus 100 hours replayed from the base model's mixture to prevent forgetting.
• 4 epochs / 57,056 updates at a reduced learning rate (5e-6), seeded from the base model's EMA weights.
• Yoruba tone marks enforced throughout the new text.
• Same architecture, same 3,821-token vocabulary, drop-in compatible with the base.

Results

Scored with the same ASR-in-the-loop protocol as the base model: generated clip and real human reference are both transcribed by MMS 1b-all, and we report the CER gap (generated − human reference), in percentage points. Lower is better; 0 is parity with the human clip.

Headline: Hausa improves from +18.4 to +8.5. Northern Sotho (+15.5 → +7.7), Fula, Kamba and Igbo also improve. Amharic moves only slightly (+78.8 → +75.0): its problem is data quality, not training time. A few non-focus languages drift upward a little — the visible cost of a targeted booster. Averages: base +6.1, this model +7.1 across the 21 scoreable languages.

Language	Base	Boost	Δ		Language	Base	Boost	Δ
Hausa	+18.4	+8.5	−9.9		Swahili	−1.1	+0.6	+1.7
Northern Sotho	+15.5	+7.7	−7.8		Kabuverdianu	+3.6	+6.3	+2.7
Amharic	+78.8	+75.0	−3.8		Lingala	−7.7	−5.2	+2.5
Fula	+5.0	+2.6	−2.4		Luo	+2.9	+5.4	+2.5
Kamba	+6.4	+4.4	−2.0		Wolof	−3.7	+2.2	+5.9
Igbo	+3.6	+1.9	−1.7		Somali	−1.9	+6.9	+8.8
Zulu	+0.8	−0.0	−0.8		Shona	+3.9	+13.8	+9.9
Nyanja	−2.2	−3.1	−0.9		Yoruba	+30.9	+31.6	+0.7
Xhosa	−1.7	−1.7	0.0		Afrikaans	+17.4	+17.4	0.0
Oromo	+9.8	+9.8	0.0		Umbundu	+3.7	+4.7	+1.0

Twi, Pidgin, code-switched and accented English have no reliable multilingual ASR baseline, so they are evaluated by listening tests rather than this table; the improvements there (particularly Pidgin, trained on roughly six times more real speech than the base had) are the main reason this model exists. Luganda is excluded: the recognizer mis-hears even the human reference there.

Which to use? Pick this model for Hausa, Pidgin, Twi, accented English, Northern Sotho or Fula. Pick the base model if Shona, Somali or Wolof are your priority. Yoruba is equivalent on both.

Usage

pip install f5-tts==1.1.22

from huggingface_hub import hf_hub_download
from f5_tts.api import F5TTS

ckpt = hf_hub_download("travis-moore/sema-tts-boost", "sema_boost_v1.pt")
vocab = hf_hub_download("travis-moore/sema-tts-boost", "vocab.txt")

tts = F5TTS(model="F5TTS_v1_Base", ckpt_file=ckpt, vocab_file=vocab, use_ema=True)
tts.infer(
    ref_file="speaker.wav",            # 5-10s clean clip of any voice
    ref_text="what the speaker says in that clip",
    gen_text="Text to speak, in any supported language.",
    file_wave="out.wav",
    nfe_step=32, speed=0.9, seed=42,
)

Tips:

• A clean, calmly-paced 5–10 s reference gives the best clone; the model copies the reference's rhythm.
• The model honors full stops much more than commas — prefer short sentences.
• With f5-tts==1.1.22 and this extended vocab, patch one line before batch inference: in f5_tts/model/utils.py, change padding_value=-1 to padding_value=0.

Limitations

• Amharic remains far behind its human baseline even after boosting; the bottleneck is available data quality, and it should be treated as experimental.
• Yoruba scores are noisy across checkpoints; listen before choosing.
• A booster trades a little generality for its targets: Shona, Somali and Wolof score somewhat worse here than in the base model.
• Scores are single-sentence ASR probes: indicative, not exhaustive MOS studies.
• Best English/Twi/Pidgin results come from West African voice references.

Safety

This model clones voices from a few seconds of audio. The Apache 2.0 license permits broad use, but impersonating a real person without their consent, deception, and fraud are explicitly against the intent of this release. Only clone voices you have the right to use, and disclose synthetic audio where listeners could reasonably be misled.

Acknowledgements

Built on sema-tts-afro, OpenF5-TTS and the F5-TTS codebase (paper). Scoring uses MMS.