udgamlabs/Shruti-ONNX

UDGAM LABS · Speech-to-text models · 55M parameters · 32 MB per language

Shruti ONNX

Shruti turns speech into text, offline, one small model per language. This repo is those models for the runtimes you already use: ONNX Runtime, the browser (onnxruntime-web) and sherpa-onnx (Android, iOS, desktop). Each folder holds the released model's 4-bit weights, written with standard ONNX operators. Each model covers its language, English, and the two mixed, with English words in Latin script. Built on Cactus Whistle.

Quick start

Python (ONNX Runtime). Use the language's name or code:

pip install shruti-onnx

from shruti_onnx import Shruti

asr = Shruti.load("urdu")                                    # or "ur"; add variant="int8" for low memory
print(asr.transcribe("clip.wav")["text"])                    # language detected
print(asr.transcribe("clip.wav", language="en")["text"])     # set English: better for Indian-accented English

Or from the command line: shruti-onnx clip.wav --lang urdu.

Browser or Node (onnxruntime-web):

npm i shruti-onnx onnxruntime-web

import * as ort from "onnxruntime-web";
import { createShruti, decodeWav } from "shruti-onnx";

const asr = await createShruti({ ort, language: "urdu" });
const samples = decodeWav(await (await fetch("clip.wav")).arrayBuffer());   // or a 16 kHz Float32Array from the mic
console.log((await asr.transcribe(samples)).text);

sherpa-onnx (Python shown; the same files work in sherpa-onnx's Android, iOS and C++ APIs, as a Moonshine model):

import sherpa_onnx, soundfile as sf

d = "ur/sherpa-onnx/"
rec = sherpa_onnx.OfflineRecognizer.from_moonshine(
    preprocessor=d + "preprocess.onnx", encoder=d + "encode.onnx",
    uncached_decoder=d + "uncached_decode.onnx", cached_decoder=d + "cached_decode.onnx",
    tokens=d + "tokens.txt", num_threads=2)
samples, rate = sf.read("clip.wav", dtype="float32")         # 16 kHz mono
s = rec.create_stream(); s.accept_waveform(rate, samples); rec.decode_stream(s)
print(s.result.text)

Which files

Every language has the same three folders. MANIFEST.json lists each file's size and sha256.

Folder	For	Size
<code>/onnx/	ONNX Runtime and browsers: the smallest download. On Linux x86, use this one	32 MB
<code>/onnx-int8/	servers and low-memory machines on ARM: about 120 MB of memory after loading, measured on Hindi	53 MB
<code>/sherpa-onnx/	sherpa-onnx on Android, iOS and desktop, search inside the model	54 MB

Results

We compare each build to the same model in the original engine, on that language's own test sets, against both references where they differ, plus English (LibriSpeech and Svarah). Word error rate, paired 95% intervals over clips. MIXED means accuracy held on every main set and a named part did not. FAIL, where a language carries it, means at least one main set missed on both references for a build, in either direction; we publish those with the lines below saying so. No language here met every one of our six rules, so none is labelled PASS. Every language also misses the rule that sherpa-onnx must match Python on every clip, mostly on two English clips of 30 s or more, where the packages cut audio and sherpa-onnx does not. On Marathi, Manipuri and Nepali the share of Latin letters in sherpa-onnx's output also differs from the engine's by more than a point. The table names each set where a build's interval excludes 0. A positive change means more errors than the engine. Full tables: RESULTS.md.

Language	Code	Check	Sets outside, onnx/	Sets outside, onnx-int8/	Sets outside, sherpa-onnx/
Assamese	as	MIXED	none	none	Svarah, English, set / detected: -0.80 [-1.50, -0.19]
Bengali	bn	MIXED	none	none	Svarah, English, set / detected: +0.90 [+0.21, +1.66]
Bodo	brx	MIXED	none	DiarBench, romanised: +1.44 [+0.18, +3.07]	none
Dogri	doi	FAIL	DiarBench, own script: +1.88 [+0.40, +3.29]; DiarBench, romanised: +1.46 [+0.22, +2.83]	IndicVoices, romanised: -1.17 [-2.73, -0.02]	none
Gujarati	gu	MIXED	LibriSpeech, English, set / detected: -0.16 [-0.28, -0.04]	none	none
Hindi	hi	FAIL	DiarBench, own script: +1.19 [+0.09, +3.14]; DiarBench, romanised: +1.19 [+0.03, +3.17]	DiarBench, own script: +1.25 [+0.14, +3.21]; DiarBench, romanised: +1.22 [+0.04, +3.26]	DiarBench, own script: +1.18 [+0.06, +3.12]
Kannada	kn	FAIL	FLEURS, own script: +0.65 [+0.03, +1.29]	FLEURS, own script: +1.20 [+0.53, +1.97]	none
Konkani	kok	FAIL	IndicVoices, own script: -0.56 [-1.07, -0.09]; IndicVoices, romanised: -0.74 [-1.49, -0.16]	IndicVoices, own script: -0.68 [-1.25, -0.15]; IndicVoices, romanised: -0.92 [-1.73, -0.27]	IndicVoices, romanised: -0.77 [-1.56, -0.12]
Kashmiri	ks	MIXED	none	none	none
Maithili	mai	MIXED	none	none	none
Malayalam	ml	MIXED	none	Svarah, English, set / detected: -0.45 [-0.92, -0.02]	none
Manipuri	mni	MIXED	DiarBench, romanised: +6.01 [+0.11, +18.87]	none	none
Marathi	mr	MIXED	DiarBench, romanised: +1.62 [+0.25, +3.32]	DiarBench, romanised: +1.38 [+0.07, +3.09]	none
Nepali	ne	MIXED	none	none	FLEURS, own script: +1.45 [+0.21, +2.94]
Odia	or	FAIL	Svarah, English, set / detected: -0.38 [-0.81, -0.03]	DiarBench, own script: +2.76 [+0.48, +5.59]; DiarBench, romanised: +3.29 [+0.58, +6.46]; Svarah, English, set / detected: -0.56 [-1.04, -0.16]	DiarBench, own script: +1.82 [+0.18, +3.79]; DiarBench, romanised: +2.37 [+0.12, +4.87]
Punjabi	pa	FAIL	FLEURS, own script: -0.51 [-0.95, -0.13]	none	FLEURS, own script: -0.61 [-1.14, -0.15]
Sanskrit	sa	MIXED	Svarah, English, set / detected: -0.47 [-0.93, -0.02]	none	none
Santali	sat	MIXED	DiarBench, romanised: +1.79 [+0.04, +4.33]	none	none
Sindhi	sd	FAIL	none	DiarBench, own script: -1.44 [-3.16, -0.30]; DiarBench, romanised: -1.65 [-3.34, -0.54]	DiarBench, own script: -1.54 [-3.48, -0.26]; DiarBench, romanised: -1.75 [-3.66, -0.57]
Tamil	ta	FAIL	none	FLEURS, own script: +0.71 [+0.15, +1.29]	none
Telugu	te	FAIL	DiarBench, own script: -0.90 [-1.83, -0.26]; DiarBench, romanised: -1.20 [-2.21, -0.41]	none	DiarBench, own script: -0.87 [-1.80, -0.15]; DiarBench, romanised: -0.92 [-1.96, -0.12]
Urdu	ur	MIXED	none	none	none

• Dogri failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. More errors than the engine: onnx/ DiarBench, own script +1.88 [+0.40, +3.29]; onnx/ DiarBench, romanised +1.46 [+0.22, +2.83]. Fewer errors than the engine, which also counts as a miss: onnx-int8/ IndicVoices, romanised -1.17 [-2.73, -0.02]. sherpa-onnx/ has no such miss.
• Hindi failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. More errors than the engine: onnx/ DiarBench, own script +1.19 [+0.09, +3.14]; onnx/ DiarBench, romanised +1.19 [+0.03, +3.17]; onnx-int8/ DiarBench, own script +1.25 [+0.14, +3.21]; onnx-int8/ DiarBench, romanised +1.22 [+0.04, +3.26]; sherpa-onnx/, language detected DiarBench, own script +1.18 [+0.06, +3.12]. One clip that loops in this build and not in the engine holds 60 of the 80 extra errors on onnx/, own script. Measured after the fact, without that clip: onnx/ +0.30 [0.00, +0.61], onnx-int8/ +0.36 [+0.04, +0.68].
• Kannada failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. More errors than the engine: onnx/ FLEURS, own script +0.65 [+0.03, +1.29]; onnx-int8/ FLEURS, own script +1.20 [+0.53, +1.97]. sherpa-onnx/ has no such miss. Measured after the fact, the gap is spread across many clips, not one: without the largest it is still +1.02 on onnx-int8/ and +0.49 on onnx/.
• Konkani failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. Fewer errors than the engine, which also counts as a miss: onnx/ IndicVoices, own script -0.56 [-1.07, -0.09]; onnx/ IndicVoices, romanised -0.74 [-1.49, -0.16]; onnx-int8/ IndicVoices, own script -0.68 [-1.25, -0.15]; onnx-int8/ IndicVoices, romanised -0.92 [-1.73, -0.27]; sherpa-onnx/, language detected IndicVoices, romanised -0.77 [-1.56, -0.12].
• Odia failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. More errors than the engine: onnx-int8/ DiarBench, own script +2.76 [+0.48, +5.59]; onnx-int8/ DiarBench, romanised +3.29 [+0.58, +6.46]; sherpa-onnx/, language detected DiarBench, own script +1.82 [+0.18, +3.79]; sherpa-onnx/, language detected DiarBench, romanised +2.37 [+0.12, +4.87]. Fewer errors than the engine, which also counts as a miss: onnx/ Svarah, English, set / detected -0.38 [-0.81, -0.03]; onnx-int8/ Svarah, English, set / detected -0.56 [-1.04, -0.16]. It also misses our character check: on 2 of 196 DiarBench clips, sherpa-onnx and Python end the text differently, so 3 letters in Python's text are not in sherpa-onnx's. No broken or replaced characters appeared. On DiarBench the share of Latin letters in sherpa-onnx's text is 34.8%, against the engine's 33.0%.
• Punjabi failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. Fewer errors than the engine, which also counts as a miss: onnx/ FLEURS, own script -0.51 [-0.95, -0.13]; sherpa-onnx/, language detected FLEURS, own script -0.61 [-1.14, -0.15]. onnx-int8/ has no such miss.
• Sindhi failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. Fewer errors than the engine, which also counts as a miss: onnx-int8/ DiarBench, own script -1.44 [-3.16, -0.30]; onnx-int8/ DiarBench, romanised -1.65 [-3.34, -0.54]; sherpa-onnx/, language detected DiarBench, own script -1.54 [-3.48, -0.26]; sherpa-onnx/, language detected DiarBench, romanised -1.75 [-3.66, -0.57]. onnx/ has no such miss.
• Tamil failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. More errors than the engine: onnx-int8/ FLEURS, own script +0.71 [+0.15, +1.29]. onnx/ and sherpa-onnx/ have no such miss.
• Telugu failed our written test, and we publish it as a marked exception. The test asks each set to match the original engine within its 95% interval, in either direction. Fewer errors than the engine, which also counts as a miss: onnx/ DiarBench, own script -0.90 [-1.83, -0.26]; onnx/ DiarBench, romanised -1.20 [-2.21, -0.41]; sherpa-onnx/, language detected DiarBench, own script -0.87 [-1.80, -0.15]; sherpa-onnx/, language detected DiarBench, romanised -0.92 [-1.96, -0.12]. onnx-int8/ has no such miss.

Where it falls short

• Repeats: a model can get stuck repeating a word or phrase. The text stays as the model wrote it, as in the engine and in sherpa-onnx. In our tests 1.2% of 4-bit clips held a repeat across all the languages we ran. loops="cut" in Python, or { loops: "cut" } in JavaScript, keeps the first copy and drops what follows it; it scored further from the engine overall, so it is off by default. Long repeats can differ by device: one Sanskrit clip of 20 repeated on our test phone and not on the Mac (140 words against 27).
• Linux x86: onnx/ wrote identical text on a Mac and on Linux x86. onnx-int8/ and sherpa-onnx/ did not, and were usually less accurate on conversation, on an AMD EPYC without VNNI. Chips with VNNI are untested. Use onnx/ on x86. Table.
• Memory in ONNX Runtime: the 32 MB onnx/ build needs about 1 GB once loaded, because ONNX Runtime expands the 4-bit weights (measured on Hindi). Use onnx-int8/ or sherpa-onnx/ on ARM where memory matters.
• Indian-accented English with the language detected is often taken for the model's own language: 51-78% of Svarah clips across these languages, with more errors than setting English. The Python and JavaScript packages can set English; sherpa-onnx/ always detects.
• Long audio: the Python and JavaScript packages cut it into 30-second windows. With sherpa-onnx, split audio longer than 30 seconds with a voice-activity detector first: unsplit, the transcript can start repeating itself.
• On a phone the original engine is faster and lighter. Use these builds where your app already runs ONNX Runtime or sherpa-onnx.
• Small differences between builds and devices: text can differ by a letter or a word between a Mac and a phone, with similar accuracy overall.
• Numbers come out as spoken words.
• Names it never heard are spelt by sound, and far-away or speaker-played audio throws it: speak close to the device.
• No word-by-word streaming in the sherpa-onnx pack: the text arrives when the search for the clip ends.
• Not tested: iPhone, Windows, phones other than a Galaxy F62, browsers other than Chromium on a Mac, or a browser's microphone.

How it was made

These are the released models' weights: the same 4-bit values in the same groups, stored as standard ONNX operators instead of Cactus's file format, so no accuracy is lost to a second quantisation. Decoding is 5-beam search; in the sherpa-onnx pack the search runs inside the decoder graph.

Intended use and limitations

Offline speech-to-text in apps, browsers and devices: voice commands, notes, captions, forms. Check its output before acting on it in anything safety-critical.

Licence

Apache-2.0, like Cactus Whistle; keep the NOTICE file.

---

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com