w2v-bert-2.0-lusoga-main-best

A Lusoga automatic speech recognition (ASR) model, fine-tuned from facebook/w2v-bert-2.0 on the WAXAL Lusoga speech corpus only.

Model description

facebook/w2v-bert-2.0 — a large-scale, multilingual self-supervised speech encoder pretrained with a BERT-style masked prediction objective — is used as the backbone, with a from-scratch character-level CTC (Connectionist Temporal Classification) head fine-tuned specifically for Lusoga.

Text casing note: this model's vocabulary was built with NFKC-only normalization — case, punctuation, and diacritics from the source transcriptions are preserved as-is. This matches how the WAXAL competition (Zindi) actually scores submissions: raw, unnormalized WER/CER, where case and punctuation mismatches count as errors.

No language model: unlike the Luganda -main-best models, there is currently no matching KenLM language model for Lusoga. Decoding is greedy (argmax) only — see How to use below.

Training data

Source	Role
google/WaxalNLP (Lusoga config)	train split used for training; validation split held out untouched, used only to monitor training (early stopping / checkpoint selection)

Unlike the Luganda -main-best model, no additional community datasets were used here — training data is WAXAL only.

Training procedure

• Base model: facebook/w2v-bert-2.0
• Architecture: Wav2Vec2BertForCTC, add_adapter=True
• Processor: Wav2Vec2BertProcessor — SeamlessM4TFeatureExtractor for audio features + a Wav2Vec2CTCTokenizer built from scratch on the combined training + validation transcriptions (character-level vocabulary, case and punctuation preserved, NFKC Unicode normalization only, | as the word delimiter, [PAD] doubling as the CTC blank token)
• Sample rate: 16 kHz mono
• Hardware: single RTX PRO 6000
• Epochs: 10 (with early stopping, patience 5, on validation WER)

Hyperparameter	Value	Rationale
Learning rate	3e-5	A much higher rate (e.g. 1e-3) is too aggressive for full fine-tuning of a model this size
Effective batch size	32 (per-device 4 × grad-accum 8)	Batch size 1 gives very noisy gradients at this model scale
Checkpoint selection	best-by-WER	load_best_model_at_end + early stopping
Dropout	0.05 (attention / hidden / feature-projection)	Non-zero regularization appropriate for this dataset size
Weight decay	0.01	Standard AdamW regularization
LR schedule	cosine, 10% warmup	Gentler decay than linear, avoids an abrupt ramp-down
Precision	fp16, gradient checkpointing	Memory efficiency

• Data filtering: clips whose transcript is too long for CTC to align within the available encoder output length ("CTC-impossible" clips, roughly output_steps < 2 * label_length) are dropped from both train and validation before training
• Seed: 42 (deterministic — same seed for Python/NumPy/PyTorch/CUDA)

How to use

Greedy (argmax) decoding only — there is currently no matching KenLM language model for Lusoga.

import torch
import librosa
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor

MODEL_ID = "keystats/w2v-bert-2.0-lusoga-main-best"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]

print(transcription)  # cased, punctuated Lusoga text (to whatever extent seen in training)

Intended uses & limitations

• Intended for transcribing spoken Lusoga audio into cased, punctuated text (to whatever extent case/punctuation exist in the training transcriptions).
• As a CTC-based model, it assumes single-speaker, forward-only audio and has no mechanism for overlapping speech from multiple speakers.
• Trained only on WAXAL Lusoga data — acoustic conditions, recording quality, and dialectal coverage reflect that single source, unlike the Luganda models which pool multiple datasets.
• No language model is available for this checkpoint — output is from greedy decoding only, which typically scores somewhat worse than LM-assisted beam search.
• Raw-text WER/CER (with case and punctuation counted as errors) will read higher than a normalized-text comparison of the same underlying transcription quality — this is expected and matches how the source competition (Zindi/WAXAL) actually scores submissions.

Citation

If you use this model, please cite the training/fine-tuning work and the underlying dataset:

@misc{keystats_wav2vec2bert_lusoga_best,
  title={w2v-bert-2.0-lusoga-main-best: A Lusoga ASR model fine-tuned from facebook/w2v-bert-2.0 on raw text},
  author={keystats},
  year={2026},
  howpublished={\url{https://huggingface.co/keystats/w2v-bert-2.0-lusoga-main-best}}
}

@misc{waxal,
  title={WAXAL: A Multilingual African Speech Dataset},
  author={Google},
  howpublished={\url{https://huggingface.co/datasets/google/WaxalNLP}}
}

@inproceedings{w2vbert2,
  title={Seamless: Multilingual Expressive and Streaming Speech Translation},
  author={Seamless Communication and others},
  year={2023},
  howpublished={\url{https://huggingface.co/facebook/w2v-bert-2.0}}
}