antonbanderi/fjale

Duovid Whisper — Albanian (CoreML 6-bit · whisper.cpp GGML q5_0)

On-device Albanian (Shqip) speech recognition for the Duovid iOS app, packaged for WhisperKit (Core ML). Variant folder: duovid_whisper-large-v3-turbo-sq_pal6bit_589MB (~593 MB).

The Duovid Android app uses the whisper.cpp (GGML) build of the same weights: ggml/ggml-large-v3-turbo-sq-q5_0.bin (574 MB) — see whisper.cpp (GGML) build for Android.

Attribution

• Fine-tune: Flutra/whisper-large-v3-turbo-sq-v2 by Flutra — OpenAI Whisper large-v3-turbo fine-tuned for Albanian. All credit for the Albanian accuracy goes to this work.
• Base model: OpenAI Whisper large-v3-turbo — MIT License, Copyright (c) 2022 OpenAI.
• Training data (per the fine-tune's card): Mozilla Common Voice 19, Albanian (Kushtrim/common_voice_19_sq) — CC0 1.0 (public domain).
• Runtime (iOS): WhisperKit by Argmax, Inc. — MIT License.
• Runtime + converter (Android / GGML): whisper.cpp by Georgi Gerganov and contributors — MIT License. The GGML file embeds the 128-bin mel filterbank from openai/whisper (whisper/assets/mel_filters.npz, MIT) and the fine-tune's own vocab.json (the standard Whisper vocabulary).
• Tokenizer: tokenizer.json / tokenizer_config.json copied unchanged from openai/whisper-large-v3-turbo (MIT); the fine-tune uses the same vocabulary.

What we changed

Only the packaging — the weights are Flutra's fine-tune:

1. Converted to Core ML with Argmax's whisperkittools (MelSpectrogram, AudioEncoder, TextDecoder as compiled .mlmodelc).
2. Compressed with 6-bit palettization of the encoder/decoder weights (≈ 1.6 GB → ~589 MB). A 4-bit palettized build was also tried and rejected (> 100 % WER).

Evaluation

Real Albanian speech, 7 speakers, ~7 minutes, run through WhisperKit 1.1.1 with Duovid's decoding settings (forced language sq, word timestamps, temperature 0 with fallback, VAD chunking, no prompt). WER/CER after lower-casing and removing punctuation (diacritics kept).

Model	WER (all)	WER on unseen speech (VoxForge)	WER (Common Voice test)	CER
This model (Flutra fine-tune, CoreML 6-bit, 589 MB)	11.3 %	18.8 %	2.8 %	3.1 %
Stock openai_whisper-large-v3-v20240930_turbo_632MB	42.8 %	44.9 %	40.3 %	11.1 %

The Common Voice clips may overlap the fine-tune's training distribution, so the VoxForge column (speakers and recordings the fine-tune never saw) is the honest figure: ~19 % vs ~45 % WER. 99.8 % of word timestamps fell inside the correct sentence window; real-time factor ≈ 0.04 on an M-series Mac.

Usage (WhisperKit)

let folder = try await WhisperKit.download(variant: "duovid_whisper-large-v3-turbo-sq_pal6bit_589MB", from: "antonbanderi/fjale")
let kit = try await WhisperKit(WhisperKitConfig(model: "duovid_whisper-large-v3-turbo-sq_pal6bit_589MB", modelRepo: "antonbanderi/fjale",
                                                modelFolder: folder.path, download: false))
let result = try await kit.transcribe(audioPath: path,
                                      decodeOptions: DecodingOptions(language: "sq", usePrefillPrompt: true,
                                                                     detectLanguage: false, wordTimestamps: true,
                                                                     chunkingStrategy: .vad))

whisper.cpp (GGML) build for Android

File	ggml/ggml-large-v3-turbo-sq-q5_0.bin
Size	574,041,195 bytes (~574 MB)
SHA-256	383088bcdd2bd4fd444cdcc8cf1c43404e33e547ae35e85047992e1a2b605449
Quantization	q5_0 (whisper.cpp whisper-quantize; same size/format as stock ggml-large-v3-turbo-q5_0.bin)
Built with	whisper.cpp v1.9.5

How it was made (weights unchanged apart from quantization):

1. models/convert-h5-to-ggml.py from whisper.cpp v1.9.5 on the Hugging Face transformers checkpoint Flutra/whisper-large-v3-turbo-sq-v2 (fp32 safetensors) with the mel filters from openai/whisper → GGML f16 (1.62 GB).
2. whisper-quantize ggml-model.bin ggml-large-v3-turbo-sq-q5_0.bin q5_0.

Evaluation — the same 7 clips as above (4 Common Voice, 3 VoxForge speakers, ~7 min), transcribed with whisper-cli -l sq (no prompt), whisper.cpp v1.9.5 on an M-series Mac; same normalisation as above (lower-case, punctuation removed, diacritics kept; the "no-dia" column also folds ë→e, ç→c). Beam search = whisper-cli default (beam 5, best-of 5); greedy = -bs 1 -bo 1; "CPU" = -ng (no GPU, like phones).

Model (GGML)	Size	Decoding	WER (all)	WER no-dia	WER unseen (VoxForge)	WER Common Voice	CER
Fine-tune f16 (reference)	1.62 GB	beam 5	10.2 %	9.6 %	17.8 %	1.6 %	2.9 %
Fine-tune q8_0	874 MB	beam 5	10.2 %	9.4 %	18.5 %	0.8 %	3.0 %
Fine-tune q8_0	874 MB	greedy	10.6 %	10.0 %	19.2 %	0.8 %	2.9 %
Fine-tune q5_0 (this file)	574 MB	beam 5	10.4 %	9.6 %	18.8 %	0.8 %	2.9 %
Fine-tune q5_0 (this file)	574 MB	greedy	10.4 %	9.6 %	18.8 %	0.8 %	3.0 %
Fine-tune q5_0 (this file)	574 MB	beam 5, CPU	10.2 %	9.4 %	18.1 %	1.2 %	2.9 %
Fine-tune q5_0 (this file)	574 MB	greedy, CPU	10.9 %	10.2 %	19.5 %	1.2 %	3.1 %
Stock ggml-large-v3-turbo-q5_0.bin	574 MB	beam 5	37.6 %	35.0 %	35.5 %	39.9 %	9.6 %
Stock ggml-large-v3-turbo-q5_0.bin	574 MB	greedy	40.0 %	37.8 %	38.0 %	42.3 %	10.2 %

q5_0 is within 0.2 WER points of q8_0 (and of f16) at two thirds of the size, so it is the published build. It matches the Core ML 6-bit build (11.3 % / 18.8 %). As above, the VoxForge column is the honest number for speech the fine-tune never saw (~19 % vs ~36 % for stock large-v3-turbo); 7 clips is a small sample, so treat differences under ~1 point as noise.

Usage (whisper.cpp)

hf download antonbanderi/fjale ggml/ggml-large-v3-turbo-sq-q5_0.bin --local-dir .
# 16 kHz mono WAV in; -l sq forces Albanian
./build/bin/whisper-cli -m ggml/ggml-large-v3-turbo-sq-q5_0.bin -l sq -f audio.wav -ml 1 -sow -oj

From C/C++ (e.g. an Android NDK/JNI bridge): whisper_init_from_file_with_params(path, cparams), then whisper_full(ctx, params, pcm, n) with params.language = "sq", params.detect_language = false, params.token_timestamps = true for word timings. Direct download URL (public, no token): https://huggingface.co/antonbanderi/fjale/resolve/main/ggml/ggml-large-v3-turbo-sq-q5_0.bin.

License

The fine-tuned weights belong to their author, Flutra, and are subject to the terms set by that author (see the fine-tune's page). OpenAI's MIT licence covers only the base Whisper model, not these weights, so this repository does not grant an MIT (or any other) licence for them.

This applies equally to the Core ML build and to the GGML file in ggml/: converting and quantizing the weights does not change their terms.

The base model (OpenAI Whisper), the tokenizer files and the mel filterbank are MIT, Copyright (c) 2022 OpenAI. whisper.cpp (runtime and conversion tooling, not included in this repository) is MIT, Copyright (c) 2023-2026 The ggml authors. The Common Voice training data is CC0. Provided "as is", without warranty of any kind.