adidsh/indic-transcribe-flex-int8-onnx

Indic Transcribe Flex INT8

100% On-Device, Quantized INT8 ONNX Speech Recognition Package for 27 Indian Languages

This repository provides a modular, weight-quantized, Microsoft ONNX Runtime-compatible derivative of Bodhan AI's Indic-Transcribe-Flex model, specifically optimized for private, offline on-device speech-to-text inference in on-device mobile applications.

---

Model Lineage & Provenance

• Derivative Author: Community Contributor (2026)
• Direct Upstream Model: bodhan-ai/indic-transcribe-flex
• Upstream Commit SHA: d1f27e693a2d0dd5735492908720003bdac81712
• Underlying Base Architecture: nvidia/canary-1b-v2
• Base Architecture License: Creative Commons Attribution 4.0 International (CC-BY-4.0)
• Upstream License: Indic Open Model License v1.0

This model is a quantized derivative of bodhan-ai/indic-transcribe-flex. It is not presented as an original foundation model. All upstream rights, architectures, and licenses are strictly preserved.

---

Quantization Specifications

• Quantization Type: Dynamic Activation + Static Weight INT8 (W8A8 Dynamic Linear)
• Quantized Projections: 530 nn.Linear layers (289 in Encoder, 241 in Decoder) mapped to ONNX MatMulInteger operations
• Weight Initializers: 1,060 INT8 weight tensors (DataType 3)
• Dynamic Activation Quantization: Performed via ONNX DynamicQuantizeLinear nodes
• Preserved High-Precision Paths: Convolution subsampling blocks, LayerNorm layers, Rotary Positional Embeddings, and residual additions remain strictly in FP32 (DataType 1)
• Footprint Comparison:
• Upstream FP32 Checkpoint: 4.55 GB (model.safetensors)
• Quantized INT8 ONNX (encoder.onnx + decoder.onnx): 1.45 GB (1,570,095,829 bytes)
• Storage Reduction: 68.1%

---

Supported Languages (27 Languages)

The model supports speech transcription and language identification across 27 Indian languages:

• Indian English: en
• 22 Official Languages: as (Assamese), bn (Bengali), brx (Bodo), doi (Dogri), gu (Gujarati), hi (Hindi), kn (Kannada), ks (Kashmiri), kok (Konkani), mai (Maithili), ml (Malayalam), mni (Manipuri), mr (Marathi), ne (Nepali), or (Odia), pa (Punjabi), sa (Sanskrit), sat (Santali), sd (Sindhi), ta (Tamil), te (Telugu), ur (Urdu)
• Hindi Dialects & Low-Resource: bho (Bhojpuri), hne (Chhattisgarhi), bgc (Haryanvi), bhb (Bhili)

---

Package Structure

indic-transcribe-flex-int8-onnx/
├── README.md # Model Card (this document)
├── LICENSE # Full text of Indic Open Model License v1.0
├── LICENSE_DEED.md # Plain-language deed summary
├── NOTICE.md # Legal attribution for Bodhan AI and NVIDIA
├── config/
│ ├── config.json # Canary FastConformer 32L/24L dimensions
│ ├── tokenizer_config.json # Tokenizer layout & language prompt IDs
│ └── generation_config.json # Autoregressive generation parameters
├── model/
│ ├── encoder.onnx # 32-layer FastConformer Encoder (1.04 GB)
│ ├── decoder.onnx # 24-layer Transformer Decoder (427.5 MB)
│ ├── bodhan_vocab.json # 7,152 token piece array for mobile decoding (84.8 KB)
│ ├── tokenizer_multilingual.model # SentencePiece multilingual vocabulary
│ └── tokenizer_spl_tokens.model # SentencePiece special tokens vocabulary
└── metadata/
 ├── manifest.json # Versioned download manifest
 └── checksums.sha256 # SHA-256 verification sums

---

Android App Consumption Contract

On-device mobile applications consumes this repository deterministically:

1. Queries metadata/manifest.json.
2. Verifies client app version compatibility (minimum_app_version_code: 295).
3. Confirms available disk storage (> 2.0 GB free).
4. Streams model/encoder.onnx, model/decoder.onnx, and model/bodhan_vocab.json directly into temporary storage.
5. Computes streaming SHA-256 digests and validates against manifest.json.
6. Atomically moves verified models into the internal app storage path: /files/models/bodhan_flex_int8/.
7. Instantiates ONNX Runtime CPU sessions in CanaryAedEngine.kt.

---

Limitations & Usage Guidelines

1. Acoustic Conditions: Optimized for single-speaker conversational speech. Overlapping speakers or heavy background distortion may degrade output.
2. Language ID: Automatic language identification on short snippets (< 2 seconds) may exhibit lower confidence between closely related Northern Indic varieties; explicitly passing the prompt language token is recommended when the language is known.
3. Hardware Requirements: On-device INT8 CPU inference requires an ARM64-v8a processor with at least 4 GB total system RAM (2 GB free during inference).
4. Streaming: Real-time dictation is achieved via adaptive audio chunking, rather than native streaming states in the Canary AED architecture.

---

License & Legal Terms

• Model Distribution: Released under the Indic Open Model License v1.0.
• Base Architecture: NVIDIA Canary-1B-v2 released under CC-BY-4.0.
• Commercial Use: Commercial on-device self-hosted inference is permitted under the Indic Open Model License v1.0. Products exceeding 500M+ MAU or $250M+ annual revenue require a separate commercial license from Bodhan AI.
• Hosting Restriction: This repository distributes files for local on-device client inference. Providing this model as a hosted third-party API service requires written approval from Bodhan AI.
• Attribution:

"Built with Indic-Transcribe-Flex from Bodhan AI / AI4Bharat."
"Base model: Canary-1B-v2 developed by NVIDIA Corporation (CC-BY-4.0)."

---

Prohibited Uses (Section 10)

As mandated by Section 10 of the Indic Open Model License v1.0, this model and its derivatives must not be used for:

• Child sexual abuse material or content sexualizing minors.
• Conventional, chemical, biological, radiological, or nuclear weapons development.
• Mass surveillance or social-scoring systems.
• Disinformation campaigns or election manipulation.
• Automated decisions materially affecting legal rights without human oversight.
• Non-consensual voice cloning or deepfakes of real individuals.
• Robocalls, auto-dialers, or voice-phishing (vishing) schemes.
• AI companion products designed to simulate romantic intimacy or emotional dependency.

---

Citation & References

ibtex @misc{indictranscribe2026, title = {Indic-Transcribe: Built for the way India actually speaks}, author = {Bodhan AI, AI4Bharat}, year = {2026}, url = {https://bodhan.ai/research/blogs/indic-transcribe} }

• Upstream Model Card: bodhan-ai/indic-transcribe-flex
• Base Architecture: nvidia/canary-1b-v2
• Upstream License: Indic Open Model License v1.0