Inflect-Micro-v2-ONNX-WebGPU

owensong/Inflect-Micro-v2's ONNX export (owensong/Inflect-Micro-v2-ONNX) with its weights quantized for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference.

Changes from the original

onnx/duration.onnx and onnx/decode.onnx quantized without calibration (round to nearest, symmetric, blocks of 32): matrix multiplications to 4-bit MatMulNBits, embedding tables to GatherBlockQuantized, convolution weights to blocked int4 DequantizeLinear (opset 21). Activations stay float32. 38 MB → 6.5 MB.

These are modified files: their outputs are close to, but not identical to, the original model's. On eight test sentences, Whisper small.en heard the quantized model with a 10.2% word error rate (the original: 13.6%).

Use

As the original's onnx/inference_onnx.py: the duration graph, then the decoder with Gaussian latent noise; inputs, outputs and the text front end are unchanged.

Licence

Apache 2.0, as the original (LICENSE; THIRD_PARTY_NOTICES.md as the original ships it).