Request access to moore-tts-chatterbox
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
This model is part of a research benchmark on text-to-speech for Mooré. Requests are reviewed by hand, usually within a few days. Generated speech can imitate real voices, so access comes with a few conditions: respect the license (MIT), do not clone or imitate a real person's voice without their consent, say clearly that the audio is synthetic when you share it, and do not use the model for fraud, harassment, impersonation or political misinformation.
Log in or Sign Up to review the conditions and access this model content.
Mooré Chatterbox Multilingual (LoRA)
Chatterbox Multilingual V3 adapted to Mooré with a rank-64 LoRA and a new [mos] language tag, trained on 54.6 hours of read Mooré. It clones a voice from a short reference clip.
- CER 1.50 % on 150 held-out Bible verses, against 0.47 % for the real recordings and 29.38 % for the untouched base model
- Zero-shot voice cloning from a reference clip, 24 kHz output, real-time factor 0.21 on one RTX 5090
- Part of a controlled benchmark of open TTS architectures on Mooré (same data, same splits, same evaluation)
Samples
Four verses drawn at random from the held-out dev set (chapters never seen in training). For each one: this model, the untouched base model, and the reference clip of the voice being cloned.
| Text | Voice | Type | Audio |
|---|---|---|---|
| Mam sã n zẽk m zugu, yãmb tãoosda maam wa gɩgemd sẽn digd bũmbu, la y le tar y pãngã n maand maam wẽnga. | V13 | this model (CER 1.0%) | |
| Mam sã n zẽk m zugu, yãmb tãoosda maam wa gɩgemd sẽn digd bũmbu, la y le tar y pãngã n maand maam wẽnga. | V13 | ResembleAI/chatterbox, no fine-tuning (CER 25.7%) | |
| (voice prompt: a different verse by the same reader) | V13 | reference clip | |
| T'a Zeezi yeel-a: Mam sẽn gomd ne foomã, mam yaa yẽnda. | V04 | this model (CER 0.0%) | |
| T'a Zeezi yeel-a: Mam sẽn gomd ne foomã, mam yaa yẽnda. | V04 | ResembleAI/chatterbox, no fine-tuning (CER 19.2%) | |
| (voice prompt: a different verse by the same reader) | V04 | reference clip | |
| A wilga tõnd t'a yãa malɛk sẽn wa n yaas a roogẽ wã, n yeele: 'Tʋm rap Zope tɩ b tɩ bool a Sɩmo b sẽn boond me t'a Pɩyɛɛrã, | V07 | this model (CER 3.4%) | |
| A wilga tõnd t'a yãa malɛk sẽn wa n yaas a roogẽ wã, n yeele: 'Tʋm rap Zope tɩ b tɩ bool a Sɩmo b sẽn boond me t'a Pɩyɛɛrã, | V07 | ResembleAI/chatterbox, no fine-tuning (CER 23.5%) | |
| (voice prompt: a different verse by the same reader) | V07 | reference clip | |
| La sapɩlmã na n sarma kãgrã, la b na n sirga tẽngã fasɩ. | V11 | this model (CER 0.0%) | |
| La sapɩlmã na n sarma kãgrã, la b na n sirga tẽngã fasɩ. | V11 | ResembleAI/chatterbox, no fine-tuning (CER 22.2%) | |
| (voice prompt: a different verse by the same reader) | V11 | reference clip |
Usage
# pip install "chatterbox-tts @ git+https://github.com/resemble-ai/chatterbox@5de7a54" peft==0.20.0
# huggingface-cli download anyantudre/moore-tts-chatterbox --local-dir moore-cbx
# lines.jsonl: {"id": "1", "text": "...", "voice": "x", "prompt_path": "reference.wav", "prompt_text": ""}
python moore-cbx/code/synth.py --ckpt moore-cbx --manifest lines.jsonl --out_dir out --device cuda
Training
| Base model | ResembleAI/chatterbox |
| Parameters | ~800M at inference (T3 536M with a merged rank-64 LoRA, S3Gen 264M) |
| Output | 24000 Hz mono |
| Data | 54.6 h, 20,491 verses read by ~16 voices, jw.org Mooré Bible recordings (24 kHz), train chapters only; edge silences trimmed, loudness -20 LUFS |
- LoRA r64, alpha 128, dropout 0.05 on the attention and MLP projections and the speaker encoder; text embedding and text head trained in full
- New language tag
[mos], initialised from[sw](Swahili); Mooré is not one of the 23 base languages - 12,000 steps, lr 1e-4 cosine with 5 % warmup, about 26 clips per step; S3 speech tokens precomputed with the official tokenizer
- One RTX 5090, 2 GPU-hours; checkpoint chosen by CER of the eval ASR on dev150 (step 4,000)
Results
| dev150 (150 verses, 20 held-out chapters, 9 voices) | CER | WER | CER on ɛ ɩ ʋ and nasals | Speaker sim. to prompt (ECAPA) | UTMOSv2 | F0 contour dist. (st) |
|---|---|---|---|---|---|---|
| Real recordings | 0.47 % | 1.85 % | 1.32 % | 0.73 | 3.03 | 0.00 |
| ResembleAI/chatterbox, no fine-tuning | 29.38 % | 66.89 % | 77.09 % | 0.70 | 2.78 | 2.47 |
| This model | 1.50 % | 4.61 % | 3.70 % | 0.78 | 2.94 | 1.50 |
CER: anyantudre/moore-asr-eval-w2vbert, greedy CTC, never trained on these chapters. Speaker similarity: cosine between ECAPA-TDNN (speechbrain/spkrec-ecapa-voxceleb) embeddings of the audio and of the voice prompt; for the real recordings it compares two different verses by the same reader, which is the ceiling. UTMOSv2 is an English-trained naturalness predictor and is only a rough guide for Mooré. F0 contour distance: word-aligned DTW between synthesized and natural pitch, in semitones (lower is closer). Every system clones the same reference clip per verse.
Limitations
- Trained on read Bible speech: the prosody sounds like Bible reading, and other domains are untested.
- Speech tokens come from an ASR-supervised tokenizer trained on other languages, which may blur some Mooré vowel and tone contrasts.
- Inference needs the code in
code/(it adds the[mos]tag and merges the LoRA); the stock Chatterbox loader does not know the tag.
License
MIT, inherited from ResembleAI/chatterbox. The training audio comes from Bible recordings whose terms restrict redistribution; it is not included, apart from the short reference clips in samples/.
- Downloads last month
- -
Model tree for anyantudre/moore-tts-chatterbox
Base model
ResembleAI/chatterbox