rumik-oss-1 (Ira) · 2-bit ONNX for in-browser WebGPU

A browser-sized build of rumik-ai/rumik-oss-1 (speaker Ira) for onnxruntime-web / WebGPU.

folder	file	notes
`decoder/`	`model.onnx`, `model.onnx.data` (1.2 GB)	36-layer decoder, 2-bit weights (group 32, ≈2.6 effective bits, ONNX `MatMulNBits`), fp16 activations, explicit KV-cache inputs/outputs, stop head fused in
`mimi/`	`decoder_8cb_fp32.onnx` (228 MB)	Kyutai Mimi decoder; the 32-codebook ONNX export with codebooks 8–31 zeroed → exactly the 8-codebook decode
`assets/`	`tokenizer.json`, `tokenizer_config.json`, `web_vocab.json`	original tokenizer + pruned-vocabulary table

What was changed from the original

Quantization-aware fine-tuning to 2-bit (all 252 decoder matrices except layer-0 mlp.down_proj at 4-bit), distilled against the original model's logits on ≈9k Hindi/English/Hinglish sentences voiced by the original (Ira).
Vocabulary pruned from 277,404 to 48,400 rows (32,000 text tokens incl. all 256 byte tokens, 16,384 audio tokens); rare text tokens are spelled out as bytes at inference.
Converted to ONNX (custom export, verified against the PyTorch model; stop head included).

Quality (Whisper-large-v3-turbo ASR, 160 held-out sentences, WER capped at 100%)

2-bit: 23.8% overall (English 9.6%) vs original ≈22% (English ≈9%). Roughly 1 in 160 generations can loop (e.g. "sigh sigh sigh"); cap the length and/or use the stop head threshold.

Known limitations

Expression tags render mainly in English; in Hindi/Hinglish the original model also mostly just speaks the text (the 2-bit model matches it by ear).
Rare repetition loops; romanized Hinglish input is less well covered than Devanagari/English.

Use

See the demo page (index.html, app.js) from the project folder. Prompt format: <text>Ira: [<description="happy, Hindi accent, steady pace"> ]your text with <laugh>/<chuckle>/<sigh><audio>.

License

Research and non-commercial use only, under the same terms as the original: Tiny Aya Fire's CC BY-NC 4.0 with the Cohere Labs acceptable-use addendum (https://cohere.com/cohere-labs-cc-by-nc-license). Attribution: rumik.ai (rumik-oss 1), Cohere Labs (tiny-aya-fire), Kyutai (Mimi, CC BY 4.0; ONNX export by onnx-community/kyutai-mimi-ONNX). Modifications are described above. Not for commercial products, paid synthesis services or self-hosted commercial deployments.