travis-moore/sema-afro

Sema Afro — African Speech Recognition (17 languages, code-switch)

Sema Afro transcribes 17 African languages in a single open model, including the way people switch between a language and English inside one sentence. It is trained on roughly 1.6 million audio clips (on the order of 2,000 hours) of Sema's own recordings plus licensed open-source African speech, with a dedicated decoder-token slot per language so the languages reinforce rather than dilute one another.

It is the strongest deployable open model across these African languages (it beats the open baseline, OpenAI Whisper large-v3, on 12 of 15 benchmarked languages) and it ranks first on real Ghanaian Twi/English code-switch.

⚠️ Read this before you call the model: the language argument is an internal routing slot, not the ISO code

To keep 17 languages from blurring together, each language is trained under its own spare Whisper decoder token. Those slot tokens deliberately reuse other languages' codes, so they collide with the ISO codes you'd expect:

• Twi / Twi-English code-switch → yo  (not Yoruba!)
• Yoruba → su · Igbo → haw · Luganda → lo · Kinyarwanda → lb · Zulu → ba · Xhosa → tk · Wolof → tg · Oromo → sd · Fula → nn
• (Hausa ha, Swahili sw, Amharic am, Afrikaans af, Lingala ln, Shona sn, Somali so keep their normal codes.)

Passing language="yo" for Yoruba audio will silently transcribe it with the Twi model. Never pass a bare ISO code — always go through the sema_lang() helper in the Usage section (or the full slot table at the bottom).

Highlights

• 17 African languages in one checkpoint, with intra-sentence code-switch (e.g. Twi↔English). Other open models at this breadth (Meta MMS) use a separate monolingual model per language and cannot code-switch.
• Beats OpenAI Whisper large-v3 on 12 of 15 languages benchmarked, several by very large margins (Amharic 40.9 vs 110.6, Wolof 24.3 vs 58.9, Yoruba 34.5 vs 49.6). It ties Swahili and trails on Afrikaans and Somali.
• Keeps near-native English (CER 2.9) — which is exactly what lets it handle Twi↔English code-switch instead of breaking on the English spans.
• #1 on Ghanaian Twi/English code-switch — 34.2 CER on 78 human-verified clips, ahead of GhanaNLP's Khaya (34.5), OpenAI gpt-4o (39.7), Meta MMS (48.0) and Intron (78.4).
• Honest caveat: on single-language read speech, Meta MMS posts lower error on most languages, but it is CC-BY-NC (non-commercial), ships one model per language, and cannot code-switch — a research ceiling, not a deployable competitor. The apples-to-apples open+commercial comparison is Whisper large-v3, below.

Benchmarks

Metric. Collapsed Character Error Rate (CER): lowercase, strip punctuation and whitespace, Levenshtein distance / reference length. Lower is better. We report CER (not WER) because African word segmentation is unstable.

Method. Every model is scored on the same audio with the same scorer. Read-speech rows are the FLEURS test split (held out — we train only on train), first 150 clips per language. The code-switch row is 78 human-verified Twi/English clips in no model's training data. Competitors run on their own documented best config and current versions (October 2026).

Ghanaian Twi/English code-switch (78 human-verified clips)

System	CER ↓
Sema Afro (this model)	34.2
Khaya / GhanaNLP	34.5
OpenAI gpt-4o-transcribe	39.7
Meta MMS	48.0
Intron Sahara	78.4

Read speech, FLEURS test — Sema Afro vs the open baseline

Language	Sema Afro	OpenAI Whisper large-v3	Meta MMS¹	Intron¹	OpenAI gpt-4o¹
Hausa	18.5 ✅	25.7	7.2	7.7	—
Yoruba	34.5 ✅	49.6	20.8	18.6	—
Igbo	23.7 ✅	35.5	13.4	28.8	39.6
Swahili	8.0 ➖	8.0	3.6	7.9	7.3
Lingala	9.2 ✅	12.4	4.2	—	—
Luganda	16.1 ✅	20.8	7.0	28.6	14.6
Amharic	40.9 ✅	110.6	7.8	21.9	—
Afrikaans	11.4 ❌	9.7	6.6	11.0	5.2
Shona	14.7 ✅	21.1	4.6	7.4	—
Somali	32.6 ❌	31.4	13.5	—	—
Zulu	19.7 ✅	30.8	6.4	11.0	24.7
Xhosa	22.3 ✅	30.4	6.9	11.4	31.7
Wolof	24.3 ✅	58.9	12.3	22.1	22.7
Oromo	28.3 ✅	40.9	18.4	25.5	26.2
Fula	20.1 ✅	25.9	14.9	18.1	25.9
English	2.9 ✅	~4	—	—	—

✅ Sema Afro beats Whisper large-v3 · ➖ tie · ❌ Whisper wins. Record across the 15 African read-speech languages: 12 wins, 1 tie, 2 losses (and English adds a 13th win).

¹ Meta MMS, Intron and OpenAI gpt-4o are shown for context but are not commercially deployable open models (MMS is CC-BY-NC and monolingual-per-model; Intron and OpenAI are closed/non-commercial APIs). Whisper large-v3 (MIT) is the apples-to-apples open+commercial comparison.

Kinyarwanda (rw) and Twi (tw) are supported but not in FLEURS; Twi is measured on the code-switch set above. English (en) is a separate FLEURS split; on its full test (647 clips) Sema Afro scores CER 2.9. A full-split re-run of the table above (whole FLEURS test, not 150 clips) confirmed every language within ~1–3 CER, with Yoruba improving to 31.2.

Usage

pip install --upgrade transformers torch librosa

The language value is a routing slot, not an ISO code (see the warning at the top). Always resolve it through sema_lang() so you never accidentally send, say, Yoruba audio to the Twi slot.

import torch
from transformers import pipeline

# Friendly name -> internal routing slot. NEVER pass a bare ISO code as `language`.
SEMA_SLOT = {
    "twi": "yo", "akan": "yo", "code-switch": "yo", "twi-english": "yo",
    "hausa": "ha", "swahili": "sw", "amharic": "am", "afrikaans": "af",
    "lingala": "ln", "shona": "sn", "somali": "so",
    "yoruba": "su", "igbo": "haw", "luganda": "lo", "kinyarwanda": "lb",
    "zulu": "ba", "xhosa": "tk", "wolof": "tg", "oromo": "sd", "fula": "nn",
}
def sema_lang(name: str) -> str:
    try:
        return SEMA_SLOT[name.strip().lower()]
    except KeyError:
        raise ValueError(f"Unknown language '{name}'. Supported: {sorted(SEMA_SLOT)}")

asr = pipeline(
    "automatic-speech-recognition",
    model="travis-moore/sema-afro",
    torch_dtype=torch.float16,
    device="cuda:0",  # or "cpu"
)

# Twi + English code-switch
print(asr("ghana_twi_english.wav", generate_kwargs={"language": sema_lang("twi"), "task": "transcribe"}))
# Yoruba  (sema_lang maps this to the 'su' slot — NOT 'yo'!)
print(asr("yoruba.wav",            generate_kwargs={"language": sema_lang("yoruba"), "task": "transcribe"}))
# Hausa
print(asr("naija_hausa.wav",       generate_kwargs={"language": sema_lang("hausa"), "task": "transcribe"}))

Multiple files (set batch_size):

print(asr(["a.wav", "b.wav"], batch_size=8, generate_kwargs={"language": sema_lang("twi"), "task": "transcribe"}))

Long audio (over 30s) — enable chunking:

asr = pipeline("automatic-speech-recognition", model="travis-moore/sema-afro",
               chunk_length_s=30, batch_size=8, torch_dtype=torch.float16, device="cuda:0")
print(asr("long_audio.mp3", generate_kwargs={"language": sema_lang("hausa"), "task": "transcribe"}))

Timestamps:

print(asr("audio.wav", return_timestamps=True,
          generate_kwargs={"language": sema_lang("twi"), "task": "transcribe"})["chunks"])

On language selection. Always pass the language — Sema Afro does not reliably detect the language on its own (it is biased toward the Twi slot), so leaving it unset can mis-route non-Twi audio. For fully hands-off transcription, run a spoken-language-ID step first (e.g. an mms-lid classifier) and map its result through sema_lang().

Languages (17) and their routing slots

The slot is the value you pass as language (via sema_lang()), not the language's ISO code. Several slots intentionally reuse other languages' codes — read the warning at the top.

Language	slot		Language	slot		Language	slot
Twi/Akan + code-switch	yo		Hausa	ha		Zulu	ba
Swahili	sw		Yoruba	su		Xhosa	tk
Amharic	am		Igbo	haw		Wolof	tg
Afrikaans	af		Luganda	lo		Oromo	sd
Lingala	ln		Kinyarwanda	lb		Fula	nn
Shona	sn		Somali	so

That is 17 languages. Each slot is a stable routing token kept distinct so the languages don't blur together.

Model details

Sema Afro is an encoder-decoder speech model (~809M parameters) that transcribes audio at 16 kHz. Each language is trained under its own decoder-token slot. It is distributed as safetensors and loads with the standard transformers automatic-speech-recognition classes.

Training data

Trained on roughly 1.6 million audio clips (on the order of 2,000 hours) across the 17 languages, combining Sema's own collected recordings with licensed open-source African speech corpora. All data is cleared for release.

Limitations

• Not the most accurate per-language read-speech model — Meta MMS leads there (non-commercial; see caveat).
• Pass the language, and pass it as a slot — automatic language detection is unreliable, and the language argument is a routing slot, not an ISO code (use sema_lang()).
• Ga (gaa) is not supported — no clean transcribed Ga training data currently exists.
• Read-speech benchmarks are FLEURS (clean, recorded); real-world noisy audio is harder.
• Table numbers are 150-clip FLEURS-test / 78-clip code-switch samples (a full-split re-run confirmed them within ~1–3 CER; English is reported on the full 647-clip split) — directional at the decimal, solid at the margin.