Foreverstorm/radiochat-laya-multilingual-onnx

Laya multilingual — ONNX, 8-bit (for in-browser moderation)

A browser-ready export of convaiinnovations/laya-multilingual (revision e4e9ddf21a7b1903b7acffd8814ad4307bf63a67), the multilingual checkpoint of Laya by Convai Innovations — a non-autoregressive decision model that answers typed questions (choice, score, yes/no) in one forward pass. All credit for the model goes to its authors; see the Laya repository. Licensed Apache-2.0, like the original.

Used by RadioChat to moderate chat messages on the user's own device.

Files

The split layout that laya-ts loads:

file	size	contents
encoder.onnx	250 MB	mmBERT-base encoder — 8-bit weight-only layers (MatMulNBits, block 32), 4-bit vocabulary table (GatherBlockQuantized, block 32)
head.onnx	28 MB	decision head — 8-bit weight-only
tokenizer.json, rl_agent_config.json		unchanged from the checkpoint

How it was made

1. laya-ts/scripts/export_onnx.py --repo convaiinnovations/laya-multilingual (laya commit 9d95567), which checks the ONNX outputs against PyTorch (max logit difference ~2e-6).
2. ONNX Runtime 1.30 MatMulNBitsQuantizer (symmetric, block 32): MatMul weights 8-bit, the token-embedding Gather 4-bit; the head's MatMuls 8-bit. Each file is saved without external data (the browser loader fetches one file per graph).

Accuracy vs the full-precision model

Measured with laya-ts on onnxruntime-web (WASM) over 80 hand-labelled chat messages × 2 moderation decisions (offensive?, spam?):

variant	encoder	agrees with full model	correct vs labels
full precision	1.17 GB	160/160	152/160
this (8-bit layers, 4-bit vocabulary, 8-bit head)	250 MB	158/160	150/160
4-bit layers	195 MB	153/160	146/160

4-bit layers cost clearly more accuracy; ONNX Runtime can't run 3-bit weights.