latentdivergence/offload-whisper-small-en-npu

whisper-small.en for the Snapdragon NPU (Offload)

openai/whisper-small.en compiled ahead of time for the Qualcomm Hexagon NPU, used by the Offload Android app (com.latentdivergence.offload) to transcribe voice memos on the phone.

• Graph: Qualcomm AI Hub Models' HTP-adapted Whisper (qai_hub_models 0.64 HfWhisper: single-head attention, convolutions for linears), split into an encoder (30 s log-mel window -> cross-attention cache) and a decoder with a 200-position static self-attention cache.
• Precision: fp16 weights and activations (no integer quantisation), fp16 inputs/outputs.
• Compiled with Qualcomm AI Hub, QAIRT 2.50, --target_runtime precompiled_qnn_onnx: each folder holds QNN context binaries (encoder.bin, decoder.bin) wrapped as ONNX EPContext models (encoder.onnx, decoder.onnx), run with ONNX Runtime's QNN execution provider on the QNN 2.50 HTP runtime. A context binary only loads on its own Hexagon generation.
• tokens.bin: the tokenizer's byte strings per token id; whisper_npu.json: shapes and special token ids.

Folder	Chip	Size
sm8550/	Snapdragon 8 Gen 2 (Hexagon v73)	629 MB
sm8650/	Snapdragon 8 Gen 3 (Hexagon v75)	629 MB
sm8750/	Snapdragon 8 Elite (Hexagon v79)	628 MB
sm8850/	Snapdragon 8 Elite Gen 5 (Hexagon v81)	630 MB

Weights: OpenAI Whisper, MIT licence. The Offload app checks every file against a pinned SHA-256 before use.