killbanhar/sravaani-onnx

SraVaani-1.0 ONNX

ONNX export of ARTPARK-IISc/SraVaani-1.0, a TDT/RNNT ASR model covering 44 Indic languages/dialects, built by SPIRE Lab + ARTPARK, IISc Bangalore (paper).

All credit for the model itself goes to the original authors. This repo only provides an ONNX conversion of the encoder + decoder_joint graphs for use with onnxruntime instead of PyTorch/TorchScript — useful for platforms where PyTorch is heavy or hard to build (e.g. Android/Termux). License follows the original: MIT.

Why this exists

The original model ships as a TorchScript checkpoint (model-asr.fp16.ts) wrapped in a transformers-compatible loader, needing torch + transformers + sentencepiece at runtime — a heavy stack on resource-constrained platforms. This repo provides the same model as two ONNX graphs (encoder.onnx, decoder_joint.onnx) runnable with just onnxruntime — smaller dependency footprint, and in testing, noticeably faster than the PyTorch CPU path (see benchmarks below).

Files

File	Size	Purpose
encoder.onnx	~1.77GB (fp32)	Conformer encoder: feats[B,128,T], feature_lengths[B] → enc_out[B,1024,T'], enc_lengths[B]
decoder_joint.onnx	~43MB	LSTM predictor + joint network, single decode step: f[1,1024,1], tgt[1,1] int32, tlen[1] int32, h_in/c_in[1,1,640] → logits[1,1,1,5006], aux, h_out, c_out
config.json		Model config (vocab_size, blank_id, durations, etc. — same as original)
tokenizer.model		SentencePiece tokenizer, unchanged from original
preproc.pt		Mel-spectrogram frontend params (window, filterbank), unchanged from original

The graphs are fp32 (CPU has no fp16 conv kernel, so export runs the model upcast to fp32 — this roughly doubles encoder.onnx's size vs the original fp16 .ts checkpoint, trading disk space for CPU-inference correctness).

Usage

Decode is a greedy TDT loop: run the encoder once per utterance, then step the decoder_joint once per encoder frame (with duration-based skipping), same as any RNNT/TDT model. See onnx_transcribe.py in the companion GitHub repo for a complete reference implementation (feature extraction + greedy decode + tokenizer, using only onnxruntime, sentencepiece, torch for the STFT frontend, and soundfile).

import onnxruntime as ort

enc_sess = ort.InferenceSession("encoder.onnx", providers=["CPUExecutionProvider"])
dj_sess = ort.InferenceSession("decoder_joint.onnx", providers=["CPUExecutionProvider"])
# feats: [1, 128, T] mel-spectrogram, feature_lengths: [1]
enc_out, enc_lengths = enc_sess.run(None, {"feats": feats, "feature_lengths": feature_lengths})
# then greedy-decode by stepping dj_sess once per encoder frame - see
# onnx_transcribe.py for the full loop

Benchmarks (Android phone, CPU only, no GPU/NPU)

Tested on a mid-range Android phone (OnePlus CPH2467, arm64) via Termux, on a 4.4s audio clip:

Path	Total inference time	Realtime factor
Original PyTorch (model.transcribe(...))	~11s (6s clip)	~1.8x — slower than realtime
This ONNX export, CPUExecutionProvider	1.35s	0.31x — faster than realtime
This ONNX export, NnapiExecutionProvider	2.32s	0.53x — faster than realtime, but slower than plain CPU

Note: NNAPI hardware acceleration was slower than plain CPU for this model in testing — the decoder_joint's many small per-timestep calls incur more NNAPI dispatch overhead than they save. Recommend CPUExecutionProvider.

Transcripts from both the original PyTorch model and this ONNX export were verified byte-identical on real speech, and numerically match to within 2.5e-5 max absolute difference on raw encoder output (float32 rounding noise, not a real discrepancy).

Export process

See export_onnx.py in the companion GitHub repo to reproduce this conversion yourself. Note: export on a normal desktop PC, not Android/Termux — Termux's PyTorch build has a libc++/libcxxabi defect that corrupts TorchScript graph attribute reading during ONNX export (unrelated to this model; a general Android/Termux PyTorch packaging issue). Plain desktop PyTorch doesn't have this problem.