mnigr/whisper-swahili-tts

Whisper Small — Fine-tuned for Swahili ASR

This model is a fine-tuned version of openai/whisper-small for automatic speech recognition (ASR) in Swahili, trained on the swa_tts configuration of the google/WaxalNLP dataset.

Model Description

Whisper is an encoder-decoder Transformer originally trained by OpenAI on large-scale multilingual, multitask supervised data for speech recognition and translation. This checkpoint takes the whisper-small base model and fine-tunes it specifically on Swahili speech-text pairs to improve transcription accuracy for the Swahili language.

• Base model: openai/whisper-small
• Language: Swahili (sw)
• Task: transcribe
• Fine-tuning framework: Hugging Face transformers (Seq2SeqTrainer)

Intended Uses & Limitations

Intended uses:

• Transcribing spoken Swahili audio into text
• Research and experimentation on low-resource African language ASR
• As a base for further fine-tuning on domain-specific Swahili audio (e.g. call center, broadcast, medical, educational content)

Evaluation

Word Error Rate (WER) was computed using the evaluate library's wer metric, with basic text normalization (lowercasing, punctuation stripping, whitespace collapsing) applied to both predictions and references before scoring.

Because no dedicated held-out test split was reserved during training (all splits were merged for training data), the WER reported for this checkpoint reflects performance on a small in-sample/sanity subset rather than a true out-of-sample benchmark. If you fine-tune your own version of this model, we'd recommend holding out a validation split (EVAL_SPLIT_FRACTION > 0) for a more reliable evaluation.

How to Use

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa

model_id = "mnigr/whisper-swa-tts-finetuned"

processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id)

# Load and resample audio to 16kHz mono
waveform, sr = librosa.load("path/to/audio.wav", sr=16000)

inputs = processor(waveform, sampling_rate=16000, return_tensors="pt")

forced_decoder_ids = processor.get_decoder_prompt_ids(language="sw", task="transcribe")
predicted_ids = model.generate(
    inputs.input_features,
    forced_decoder_ids=forced_decoder_ids,
    max_new_tokens=225,
)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Framework Versions

• transformers
• datasets
• accelerate
• PyTorch (CUDA, if available)

Citation

If you use this model, please consider citing the base Whisper paper and the WaxalNLP dataset:

@misc{radford2022whisper,
  title={Robust Speech Recognition via Large-Scale Weak Supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  year={2022},
  eprint={2212.04356},
  archivePrefix={arXiv}
}

Contact

For questions, issues, or collaboration inquiries about this model, please reach out:

• Author: mnigr
• Email: anderson8k80@gmail.com

Feel free to open a discussion on the model's Hugging Face repository as well — that's often the fastest way to get a response for technical questions.