Request access to moore-asr-eval-w2vbert
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
This model is part of a research benchmark on text-to-speech for Mooré. Requests are reviewed by hand, usually within a few days. Generated speech can imitate real voices, so access comes with a few conditions: respect the license (MIT), do not clone or imitate a real person's voice without their consent, say clearly that the audio is synthetic when you share it, and do not use the model for fraud, harassment, impersonation or political misinformation.
Log in or Sign Up to review the conditions and access this model content.
Mooré eval ASR (w2v-BERT 2.0 + CTC)
A character-level Mooré speech recognizer. We use it to score how intelligible Mooré text-to-speech systems are, and it never saw the test data.
- 0.46 % CER on the held-out dev chapters, 0.60 % on the test chapters and 0.73 % on two voices it never heard, with greedy decoding and no language model
- Trained only on the training chapters of the Mooré TTS benchmark. No dev or test chapter and no held-out voice was used, in either Bible translation.
- Decoding is plain CTC. A language model trained on the Bible would fix some TTS mistakes before they are counted, so we left it out.
Usage
import torch, soundfile as sf, soxr
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor
repo = "anyantudre/moore-asr-eval-w2vbert"
proc = Wav2Vec2BertProcessor.from_pretrained(repo)
model = Wav2Vec2BertForCTC.from_pretrained(repo).eval()
wav, sr = sf.read("sample.wav", dtype="float32")
wav = soxr.resample(wav, sr, 16000)
feats = proc(wav, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
ids = model(**feats).logits.argmax(-1)
print(proc.batch_decode(ids)[0])
Compute CER on text normalized the same way on both sides: NFC, lowercase, hyphens and apostrophes turned into spaces, punctuation removed (mtts.text.normalize_cer in the project code).
Training
| Base model | facebook/w2v-bert-2.0 |
| Parameters | ~600M |
| Input | 16 kHz mono |
| Output | 32 characters: Mooré letters (ɛ ɩ ʋ, nasal vowels ã ẽ ĩ õ ũ) plus word delimiter |
| Data | ~130 h of read Mooré Bible speech: the train split of the jw.org (24 kHz) and Bible-society (16 kHz) recordings |
- 8 epochs, AdamW, lr 5e-5 with 500 warmup steps and linear decay, effective batch 32, bf16, light time masking (p=0.05), adapter on
- Best checkpoint chosen on dev CER (step 15,000 of 17,168)
- One RTX 5090, about 4.5 hours
Results
Scored on the real recordings of the frozen benchmark sets (jw.org translation), one utterance at a time.
| Set | Utterances | Chapters | CER | WER | CER on ɛ ɩ ʋ and nasal vowels |
|---|---|---|---|---|---|
| dev | 402 | 21 | 0.46 % | 1.77 % | 1.27 % |
| test | 541 | 23 | 0.60 % | 1.93 % | 1.62 % |
| test, unseen voices V17 and V18 | 836 | 40 | 0.73 % | 2.57 % | 2.43 % |
These numbers are the floor for any TTS system scored with this model: a perfect synthesis of these sentences would still get about this CER.
The dev CER logged during training was higher (1.30 % at the released step) because it was computed on padded batches, where the padding frames add spurious characters.
Limitations
- It was trained on read Bible speech only and will do much worse on conversational Mooré. It is meant for comparing TTS systems on in-domain text.
- Mooré orthography does not mark tone, so this model cannot tell whether a TTS system got the tones right.
- The output vocabulary has no ɛ̃, ɩ̃ or ʋ̃ (letters written with a combining tilde), because they never occur in the training text.
License
These weights are released under MIT, like the base model. The training audio comes from Bible recordings whose terms restrict redistribution, so the dataset stays private.
- Downloads last month
- 2
Model tree for anyantudre/moore-asr-eval-w2vbert
Base model
facebook/w2v-bert-2.0