odens00volym/kb-whisper-coreml

KB-Whisper for WhisperKit, quantized for the Neural Engine

KB-Whisper is the National Library of Sweden's Whisper, trained on Swedish and more accurate on it than any general Whisper. As published, and in every Core ML conversion of it on the Hub, its weights are float16: 1.27 GB of encoder and 1.81 GB of decoder, each stage on its own over what the Apple Neural Engine will take. On a Mac that means the ANE compiler grinds for an hour and produces nothing, and the GPU path aborts on a recording of any length.

These are the same weights, palettized so they fit.

KBLab_kb-whisper-large_1842MB

Stage	Float16	Here
AudioEncoder	1225 MB	716 MB
TextDecoder	1731 MB	1042 MB

8-bit mixed palettization with outlier decomposition, produced with whisperkittools — --generate-quantized-variants --allowed-nbits 8 --outlier-decomp. Argmax's recipe search would not go below 8 bits for this model at any layer, so this is the mildest compression that fits.

Measured on an M1 Max, 12.2 minutes of Swedish radio, Neural Engine, second run:

	Speed	Transcript
This model	1.7× real time	Inrikespolitik så i studio ett. För Centerpartiets valberedning presenterade tre kandidater som vill ta över jobbet som partiledare. Annie Lööf lämnar i februari nästa år.
openai_whisper-large-v3-v20240930_626MB	7.5×	Inrikes politik så i studiet för … Annie Lööf hon lämnar ju i februari nästa år.

Slower than Large v3 Turbo — a 32-layer decoder against Turbo's 4 — and the one that heard "studio ett" and left out the filler words. Choose it for Swedish that will be read.

The first load on a machine is slow — Core ML compiling the model for the Neural Engine, several minutes, cached by the OS afterwards.

Using it

Any WhisperKit client: modelRepo: "odens00volym/kb-whisper-coreml", modelFolder: "KBLab_kb-whisper-large_1842MB". Built for Saga Studio, where it is the "KB-Whisper Large (Swedish)" choice under Settings ▸ Transcription.

Credits and licence

• Weights: KBLab/kb-whisper-large, KBLab, Apache-2.0. Unchanged apart from precision.
• Conversion and palettization: argmaxinc/whisperkittools and argmaxtools, Argmax Inc., MIT.

Apache-2.0, as the weights are.