hogagatech/whisper.cpp-sunbird-51

Sunbird ASR Whisper 51 — GGML / Q5_0

GGML conversions of Sunbird/asr-whisper-51-african-languages for use with whisper.cpp.

The original Sunbird model supports 51 African languages.

Models

File	Description
ggml-sunbird-51.bin	Full FP16 GGML model
ggml-sunbird-51-q5_0.bin	Q5_0 quantized version

ggml-sunbird-51-q5_0.bin was quantized directly from ggml-sunbird-51.bin.

The models contain the same Sunbird-51 model; the Q5_0 version is a quantized representation intended to reduce model size and memory usage.

Original Model

Sunbird ASR Whisper 51 African Languages

https://huggingface.co/Sunbird/asr-whisper-51-african-languages

The GGML models in this repository are conversions of the original Hugging Face model.

Usage with whisper.cpp

Q5_0 model

./whisper-cli \
    -m ggml-sunbird-51-q5_0.bin \
    -f audio.mp3 \
    -l <language>

Full model

./whisper-cli \
    -m ggml-sunbird-51.bin \
    -f audio.mp3 \
    -l <language>

For example:

./whisper-cli \
    -m ggml-sunbird-51-q5_0.bin \
    -f samples/cut.mp3 \
    -l br \
    -t 3 \
    -p 4

Conversion

The models were generated using Ubuntu 22.04 with the following process:

1. Clone whisper.cpp
2. Clone OpenAI Whisper
3. Download the Sunbird Hugging Face model
4. Add the Whisper vocab.json and added_tokens.json files
5. Build whisper.cpp
6. Convert the Hugging Face model to GGML FP16
7. Quantize the GGML model to Q5_0

Dockerfile

FROM ubuntu:22.04

ENV DEBIAN_FRONTEND=noninteractive

# 1. Install system dependencies
RUN apt-get update && apt-get install -y \
    git \
    build-essential \
    cmake \
    python3 \
    python3-pip \
    python3-venv \
    ffmpeg \
    wget \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# 2. Clone repositories
RUN git clone https://github.com/ggerganov/whisper.cpp.git . && \
    git clone https://github.com/openai/whisper.git

# 3. Setup Python virtual environment & install dependencies
RUN python3 -m venv venv
ENV PATH="/app/venv/bin:$PATH"

RUN pip install --upgrade pip && \
    pip install torch torchvision torchaudio \
        --index-url https://download.pytorch.org/whl/cpu && \
    pip install huggingface_hub transformers accelerate tiktoken safetensors

# Declare Hugging Face token
ARG HF_TOKEN
ENV HF_TOKEN=${HF_TOKEN}

# 4. Download Sunbird model
RUN python3 -c 'import os, huggingface_hub; huggingface_hub.snapshot_download(repo_id="Sunbird/asr-whisper-51-african-languages", local_dir="models/Sunbird-hf", token=os.environ.get("HF_TOKEN"))'

# 5. Download legacy vocabulary files
RUN wget -q \
    https://huggingface.co/openai/whisper-large-v3/raw/main/vocab.json \
    -O models/Sunbird-hf/vocab.json && \
    wget -q \
    https://huggingface.co/openai/whisper-large-v3/raw/main/added_tokens.json \
    -O models/Sunbird-hf/added_tokens.json

# 6. Build whisper.cpp
RUN cmake -B build && \
    cmake --build build --config Release -j$(nproc)

# 7. Convert Hugging Face model to GGML FP16
RUN python3 models/convert-h5-to-ggml.py \
    models/Sunbird-hf whisper models

# 8. Quantize GGML model to Q5_0
RUN ./build/bin/quantize \
    models/ggml-model.bin \
    models/ggml-sunbird-51-q5_0.bin \
    q5_0

WORKDIR /output

CMD ["cp", "/app/models/ggml-sunbird-51-q5_0.bin", "/output/ggml-sunbird-51-q5_0.bin"]

Reproducing the conversion

Build the Docker image:

docker build \
    --build-arg HF_TOKEN=YOUR_HUGGINGFACE_TOKEN \
    -t sunbird-51-ggml .

Run it:

docker run --rm \
    -v "$(pwd)/output:/output" \
    sunbird-51-ggml

The resulting file will be:

output/ggml-sunbird-51-q5_0.bin

whisper.cpp

These models are intended for use with whisper.cpp.

The exact behavior of language selection depends on the language-token mapping contained in the converted Sunbird model. Use the language codes supported by the model/whisper.cpp version being used.

License

The original Sunbird ASR Whisper 51 model is released under the Apache License 2.0.

This repository contains converted/quantized model files derived from that model.

See the original model repository for the authoritative license and terms:

https://huggingface.co/Sunbird/asr-whisper-51-african-languages

Credits

• Sunbird AI — ASR Whisper 51 African Languages
• OpenAI — Whisper
• whisper.cpp — GGML conversion/runtime and quantization tooling