pdnhan/whisper-small.en-mobile

Whisper small.en for mobile (ONNX)

An int8 ONNX copy of OpenAI's English-only Whisper small model, for on-device speech recognition with ONNX Runtime. The Thì Thầm app uses it for English.

It is made from Xenova/whisper-small.en at commit fa16a75f5d91e83ecb6a2ccb690f14d91ef00ca4. The only change is that the attention maps are no longer graph outputs. ONNX Runtime keeps declared graph outputs in memory, so Xenova's files need much more memory for each transcription than the model needs. The weights, the nodes and the results are unchanged.

Peak memory

Net peak memory of one transcription (both sessions loaded, the encoder, then 20 decoder steps), with ONNX Runtime 1.31.1, 4 intra-op threads and graph optimization ENABLE_ALL, measured in fresh processes:

Model	Xenova file	This copy
en small	2225 MB	1032 MB

Check against the original

On 2 FLEURS English clips, 30 greedy decoder steps from the prompt give identical tokens, and the encoder output is identical, for Xenova's file and for this copy.

Inputs and outputs

• Encoder onnx/encoder_model_quantized.onnx: input input_features, float32 [1, 80, 3000] (log-mel of 30 s of audio); output last_hidden_state.
• Decoder onnx/decoder_model_merged_quantized.onnx: inputs input_ids (int64), encoder_hidden_states, use_cache_branch (bool) and past_key_values.*; outputs logits and present.*.
• Prompt: [50257, 50362] (<|startoftranscript|>, <|notimestamps|>). Stop at 50256 (<|endoftext|>).

Credits and license

• Whisper: OpenAI, openai/whisper-small.en (Apache 2.0 on Hugging Face; the code and weights in openai/whisper are under the MIT License).
• ONNX export: Xenova, Xenova/whisper-small.en, made for transformers.js.
• This copy: made with make_mobile.py (included here for provenance; it runs from tools/whisper_en of the app's repository). License: Apache 2.0, see LICENSE, as the files it is made from.
• Used by the Thì Thầm Android app for English speech recognition.