danielgloo/swahili-drc

orpheus-3b-swc-swa06 — Swahili scripture narration voice (Orpheus-3B + SNAC)

A finetune of Sunbird/orpheus-3b-tts-multilingual (Orpheus-3B: a Llama-3.2-3B causal LM that emits SNAC 24 kHz audio codes) on 96 hours of studio Bible narration in Swahili, built to read the Congo Swahili (swc, SWC02) Old Testament. Checkpoint swa06: LoRA r=64 (merged into the base weights here), 3 epochs, trained on un-deduplicated SNAC frames — the single data-prep change that produced the most intelligible model of the seven runs in this project. The adapter/ folder holds the same weights as a 390 MB PEFT adapter.

Quick start

pip install "transformers>=4.45" torch snac soundfile numpy
python infer.py --model danielgloo/orpheus-3b-swc-swa06 \
   --text "Hapo mwanzo Mungu aliumba mbingu na inchi." --out gen_1_1.wav

infer.py (in this repo) does everything that matters: the exact prompt layout, a logits processor that keeps the output inside the SNAC token grammar, batched left-padded generation, and seeded SNAC decoding. Use it or copy it.

Prompt layout (must be exact; ids are token ids of the base tokenizer):

[128259] + tokenizer("<speaker>: <text>", add_special_tokens=True) + [128009, 128260]
→ the model emits 128261 (start of audio), 128257 (start of speech), 7-token audio frames…, 128258 (end of speech)

Audio tokens: 7 per 85.3 ms frame; slot k of a frame uses ids [128266 + 4096·k, 128266 + 4096·(k+1)). Inverse interleave for SNAC: l0 = [f0], l1 = [f1, f4], l2 = [f2, f3, f5, f6].

Speaker tag: openbible_swa_0001 is the narrator this checkpoint was selected on (30.6 h of training audio, reads Genesis–Numbers, Deuteronomy, the Gospels and Acts). Six other corpus narrators exist (openbible_swa_0000…_0006).

Decoding: temperature 0.4, top_p 0.95, repetition_penalty 1.1. Measured on 40 held-out verses × 3 seeds: T 0.4 gives CER 0.053 vs 0.068 at T 0.6 and five times fewer silent omissions. Keep repetition_penalty ≥ 1.1 — at 1.0 the model loops to the token cap on a third of verses. generation_config.json carries the true stop token (eos_token_id 128258, pad_token_id 128263); the base model's config does not.

Prompt length: one verse at a time. Verses longer than ~100 text tokens (p95 of the corpus) are silently shortened by the model: 0 % at ≤100 tokens, 10 % at 120, 67 % at 180. Split long verses at clause punctuation. The base model has the same knee.

How good is it

Measured on 178 held-out verses (books never seen in training) × 3 seeds, Whisper-large-v3-turbo character error rate against the text, compared with the human narrator's own recordings on the same instrument:

	CER	notes
human narrator (studio recording)	0.047	the floor the instrument can reach
this model (swa06)	0.0665	100 % clean end-of-speech, 0 token-grammar faults
base model in its own voice (waxal_swa_0006)	0.076
previous candidate (same recipe with frame dedupe)	0.082

Voice similarity to the narrator (WavLM x-vector cosine): 0.96 (human clips 0.98). In a 1,511-verse production render of Genesis with gates (EOS, grammar, duration band, CER ≤ 0.30, unmatched-span) the first-pass failure rate was 1.8 % and every verse passed within three attempts. Throughput: ~725 tokens/s at batch 16 on one GPU (≈ 9 h of audio per GPU-hour).

Limitations — read before using for Congo Swahili

• The training audio is Coastal/Standard Swahili (swh), not Congo Swahili. No swc audio exists in the training set. The model renders swc spellings as written (asubui, inchi, pahali, magaribi, muzuri, ikakuwa, Yawe) with intelligibility equal to swh text, but with a coastal accent. Whether that is acceptable to Kingwana listeners is an open question that a native-listener panel, not a metric, decides.
• Numerals: the corpus text writes digits and the narrator read number words; convert digits to Swahili cardinals in the prompt (render/norm.py in the project does 0–999).
• The model can, rarely (~1–2 % of verses), drop a clause with a clean ending. Gate renders with an ASR check against the text and regenerate with a new seed; duration alone does not catch it.
• Register: the narrator never read poetry or prophecy; Psalms and the prophets are out of that voice's training register (they were read by other narrators in the corpus).

Licence and attribution

The training audio is open.bible Swahili (Biblica), distributed as multilingual-tts/open-bible under CC BY-SA 4.0, so these weights are released under CC BY-SA 4.0 (share-alike). Components: Orpheus-3B / Sunbird finetune, Apache-2.0; SNAC, MIT. See ATTRIBUTION.md. If you redistribute the weights or works derived from them, keep this notice.

Training summary

Base Sunbird/orpheus-3b-tts-multilingual; LoRA r=64, α=64 on q/k/v/o/gate/up/down; 30,634 verse rows (96.4 h, 7 narrators, ≤ 29.98 s); 3 epochs, bsz 16, lr 2e-4 cosine to a 0.1 floor, warm-up 2 %; held-out books Genesis and 2 Samuel (2,201 rows) never trained on; best checkpoint by held-out audio-token cross-entropy (3.497 at epoch 2.95); merged into the base in fp32 and saved in bf16 (merge verified to 1e-4 on held-out loss).