samuelolubukun/moonshine-tiny-nigerian-pidgin

Moonshine-Tiny Nigerian Pidgin (Speech-to-Text)

This repository provides fine-tuned weights, extended dialect vocabulary, and evaluation benchmarks for UsefulSensors/moonshine-tiny (~27M parameters) adapted to Nigerian Pidgin (pcm).

Fine-tuned on authentic speech recordings from the asr-nigerian-pidgin/nigerian-pidgin-1.0 corpus on an NVIDIA A10G GPU over 3,000 optimization steps.

---

Model Specifications

Attribute	Specification
Base Architecture	UsefulSensors Moonshine (Encoder-Decoder Transformer)
Base Checkpoint	UsefulSensors/moonshine-tiny (~27M parameters)
Target Language	Nigerian Pidgin (pcm)
Audio Format	16,000 Hz, 16-bit Mono PCM
Training Steps	3,000 optimization steps (Cosine Decay, AdamW)
Hardware	NVIDIA A10G (24 GB VRAM)
Precision	Float16 / BFloat16 Mixed Precision
Real-Time Factor (RTF)	0.082x (~12.2x faster than real-time)

---

Benchmark Results (Held-Out Test Set)

Evaluated across held-out test audio recordings using standard Whisper/ASR text normalization (lowercasing, punctuation stripping, whitespace normalization):

Metric	Score
Normalized Word Error Rate (WER)	54.9%
Normalized Character Error Rate (CER)	41.8%
Real-Time Factor (RTF)	0.082x
Processing Speed	< 200ms per 4-second audio

---

Qualitative Inspection on Held-Out Samples

Below is a side-by-side comparison between the ground-truth audio and the Moonshine transcription:

Sample 1 (4.5s)

• Ground Truth: dem don open dia own secretariat for asokoro inside abuja nigeria's deputy
• Moonshine Output: dem don hope on di own celebrity al for socoro inside abuja nigeria tepu tepu tepu

Sample 2 (4.7s)

• Ground Truth: oga wilson say na lie wey dem dey lie for dem head say dem be
• Moonshine Output: oga weusin say na lie wey dem dey lie for dem head see dem be

Sample 3 (5.4s)

• Ground Truth: but my guy say na hin calling be dat so rather than quit hin just dey drop more songs
• Moonshine Output: but my girl say na im call him me dat to rather than quit pin just dey drop more sunk

---

Quickstart & Inference

import torch
import librosa
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
from transformers.models.whisper.english_normalizer import BasicTextNormalizer

repo_id = "samuelolubukun/moonshine-tiny-nigerian-pidgin"
device = "cuda" if torch.cuda.is_available() else "cpu"

# 1. Load Model & Processor
processor = AutoProcessor.from_pretrained(repo_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    repo_id,
    torch_dtype=torch.float16 if device == "cuda" else torch.float32
).to(device).eval()

# 2. Load and Resample Audio (16,000 Hz mono)
wav, sr = librosa.load("audio_sample.wav", sr=16000)
inputs = processor([wav], sampling_rate=16000, return_tensors="pt")
input_values = inputs.get("input_values", inputs.get("input_features")).to(device, dtype=model.dtype)

# 3. Dynamic Token Budget (~7 tokens per second of speech)
dur = len(wav) / sr
safe_tokens = min(90, max(25, int(dur * 7)))

with torch.inference_mode():
    pred_ids = model.generate(
        input_values=input_values,
        max_new_tokens=safe_tokens,
        repetition_penalty=1.1,
        num_beams=2,
    )

transcription = processor.batch_decode(pred_ids, skip_special_tokens=True)[0].strip()
print(f"Transcription: {transcription}")

# 4. Optional Text Normalization
normalizer = BasicTextNormalizer()
print(f"Normalized: {normalizer(transcription)}")

---

Limitations & Edge Usage

• Ultra-Compact Architecture: At ~27M parameters, this model is designed for ultra-low latency, mobile devices, and real-time voice agents. For complex long-form multi-speaker broadcast audio, larger models (Whisper Medium/Large) provide greater vocabulary capacity.
• Audio Cleanliness: The model performs best on clean near-field speech (telephony, headset, or smartphone voice notes) sampled at 16 kHz.

---

Author & Attribution

Fine-tuned and benchmarked by Samuel Olubukun:

• Website: samuelolubukun.com
• GitHub: @samolubukun
• Hugging Face: @samuelolubukun

---

Dataset Attribution

Trained using data from asr-nigerian-pidgin/nigerian-pidgin-1.0. We express our sincere gratitude to the dataset curators for making speech corpora accessible for African language research.