Indic-Transcribe Flex — ONNX port
Built with Indic-Transcribe-flex from Bodhan AI / AI4Bharat. Derived from NVIDIA Canary-1B-v2. This is a format conversion, not a fine-tune.
Pinned upstream revision: ecba9bac9013c0340fe814b336897ee51ed80be9.
Made for Muesli on Windows, which runs it through ONNX Runtime in C#. The graphs are ordinary ONNX and can be used with any ONNX Runtime 1.24 or newer.
Variants
| File | Precision | Runs on | Size |
|---|---|---|---|
indic-transcribe-flex-fp16.tar |
FP16 weights, FP32 mel front end | CUDA GPU | 2.45 GB |
variants/int8/indic-transcribe-flex-int8.tar |
8-bit weights (MatMulNBits, 32-wide blocks, symmetric) | CPU | 1.71 GB |
Each tar holds one directory with encoder.*.onnx, decoder.*.onnx (each with one external
.onnx.data file), tokenizer.json, the licence and the notice.
Status
Checked against the upstream PyTorch wrapper (IndicTranscribe) in native, mixed and romanised modes on twelve clips of 1.6 to 14.2 seconds: two English clips synthesised with Windows text-to-speech, and five Hindi and five Tamil clips from the FLEURS dev set. The FP32 export reproduces all 36 upstream transcripts exactly. Identified languages match on every clip for all variants. Quantisation moves a few characters: FP16 differs by 1 character edit in 4,755 (one romanised Tamil word) and INT8 by 5 (two words in native mode, one romanised clip), measured in Python. Through Muesli's C# runtime (ONNX Runtime 1.24) both variants stay within its 0.5% character-edit check.
These are small parity checks against the upstream model, not a measure of multilingual quality, WER, robustness or production readiness.
Graphs
encoder:audio(1, samples) float32, 16 kHz mono →cross_k,cross_v(24 layers, 8 heads, frames, 128). The upstream mel front end (pre-emphasis, STFT as a fixed convolution, mel filterbank, log, per-feature normalisation) is inside the graph, followed by the FastConformer encoder and every decoder layer's cross-attention key/value projection.decoder:input_ids(1, S),past_k/past_v(P, 24, 8, 128),cross_k,cross_v→logits(1, 7152) for the last position andnew_k/new_v(S, 24, 8, 128) to append to the self-attention cache. Run the 10-token prompt first, then one token per step.tokenizer.json: token pieces, the special-token count, EOS/pad ids, the ITN and romanised prompt tokens, per-language prompts and language tokens for identification.
Prompt: [7, 4, 18, L, L, 5, 9, 11, 13, 15] with L the language token. Mixed (code-mixed) output sets position 6 to the ITN token (8); romanised output sets position 7 to 10.
Language identification decodes the first three prompt tokens and compares the logits of the
language tokens. Audio shorter than one second is centred in one second of silence, as upstream.
Decode greedily until EOS (3) or pad (2).
On CUDA, use ONNX Runtime's basic graph optimisations: with ONNX Runtime 1.24 the extended fusions corrupt the FP16 encoder's output.
Reproduction
tools/ holds the scripts: bodhan_export.py (PyTorch → FP32 ONNX, opset 17),
bodhan_quantize.py (int8 and fp16), bodhan_tokenizer.py and bodhan_check.py (parity
against the upstream PyTorch wrapper). Export used PyTorch 2.14 (CPU), transformers 5.19,
onnx 1.23 and onnxruntime 1.31. This repository contains no evaluation recordings.
Muesli
Muesli for Windows lists this model as Bodhan Flex under its experimental models, with a language picker including Auto-detect. An Output picker chooses native, mixed or romanised script; mixed is the default.
License and attribution
The converted model remains under the unmodified Indic Open Model License v1.0 included here.
Upstream NVIDIA Canary-1B-v2 is under CC BY 4.0. Preserve both attributions and the upstream
notices. See NOTICE.md and the complete license.
Sources: Bodhan model, NVIDIA base model.