indic-asr-onnx

INT8-quantized Whisper models in sherpa-onnx format, packaged for on-device speech recognition.

Packs

Each pack contains encoder.int8.onnx, decoder.int8.onnx, and tokens.txt, and is downloaded independently at runtime.

Pack	Languages	Size	Source model
whisper-tiny	en, de, fr, es, it, pt, nl, pl (+90 more)	99 MB	openai/whisper-tiny
whisper-small-ta	Tamil	358 MB	vasista22/whisper-tamil-small
whisper-small-hi	Hindi	358 MB	vasista22/whisper-hindi-small
whisper-small-ml	Malayalam	358 MB	kavyamanohar/whisper-small-malayalam
whisper-tiny-te	Telugu	99 MB	vasista22/whisper-telugu-tiny
whisper-tiny-kn	Kannada	99 MB	vasista22/whisper-kannada-tiny

whisper-tiny is the multilingual base model — it serves all European languages from a single download. The Indic packs are language-specific fine-tunes, which substantially outperform stock Whisper on those languages.

Structure

models/
├── whisper-tiny/
│   ├── encoder.int8.onnx
│   ├── decoder.int8.onnx
│   └── tokens.txt
├── whisper-small-ta/
├── whisper-small-hi/
├── whisper-small-ml/
├── whisper-tiny-te/
└── whisper-tiny-kn/

Usage

Python (sherpa-onnx)

import sherpa_onnx

recognizer = sherpa_onnx.OfflineRecognizer.from_whisper(
    encoder="models/whisper-small-ta/encoder.int8.onnx",
    decoder="models/whisper-small-ta/decoder.int8.onnx",
    tokens="models/whisper-small-ta/tokens.txt",
    language="ta",
    task="transcribe",
    num_threads=2,
)

stream = recognizer.create_stream()
stream.accept_waveform(16000, samples)   # float32 PCM, 16 kHz mono
recognizer.decode_stream(stream)
print(stream.result.text)

Android (Kotlin)

val config = OfflineRecognizerConfig(
    featConfig = FeatureConfig(sampleRate = 16000, featureDim = 80),
    modelConfig = OfflineModelConfig(
        whisper = OfflineWhisperModelConfig(
            encoder = "$packDir/encoder.int8.onnx",
            decoder = "$packDir/decoder.int8.onnx",
            language = "ta",
            task = "transcribe",
        ),
        tokens = "$packDir/tokens.txt",
        numThreads = 2,
    )
)
val recognizer = OfflineRecognizer(config)

Export

Exported with sherpa-onnx's scripts/whisper/export-onnx.py, then dynamically quantized to INT8.

HuggingFace fine-tunes were first converted to OpenAI Whisper checkpoint format ({"dims": ..., "model_state_dict": ...}) so the export script could load them.

Note: on PyTorch ≥ 2.9 the export script requires dynamo=False to be passed to torch.onnx.export — the dynamo-based exporter fails on Whisper's data-dependent positional embedding indexing.

Credits

• Fine-tuned Indic models by vasista22 (Speech Lab, IIT Madras — funded by Bhashini, MeitY, Govt. of India) and kavyamanohar
• Base models by OpenAI (Whisper)
• Export tooling by k2-fsa (sherpa-onnx)

License

Apache 2.0, inherited from the upstream models.