mosiah/fluid-speech-mini-onnx-webgpu

Fluid Speech Mini (ONNX, WebGPU)

A copy of FluidVoice-Labs/fluid-speech-mini-onnx (revision fc59262) changed so that it runs in a browser on ONNX Runtime Web's WebGPU execution provider.

Fluid Speech Mini by FluidVoice. Built on NVIDIA Parakeet TDT 0.6B v3 and Moondream Parakeet Ultra. Licensed under CC BY 4.0; see LICENSE and NOTICE. For what the model is, how accurate it is, and how to call it, read the original model card.

What was changed

Only encoder.onnx and encoder.onnx.data. Every other file is byte for byte the original.

The original encoder's MatMulNBits nodes store their zero points as floats. ONNX Runtime Web's WebGPU kernel accepts only integer zero points, so the original falls back to WebAssembly. In this copy each such node has an integer zero point plus a correction term added to its output, which is algebraically the same computation:

• largest relative difference from the original encoder's output: about 8e-8;
• the same transcripts as the original on our test clips.

The quantised weights are unchanged (2 and 4 bits, as in the original).

Using it

Use it with ONNX Runtime Web and the webgpu execution provider. Load encoder.onnx together with encoder.onnx.data. Versions that worked in our tests: 1.24.3, 1.26, 1.27, 1.29 and 1.30. 1.25 and 1.28 did not.

Measured on an Apple M-series laptop in Chromium, an 8.4-second clip transcribed in about 1.9 s on WebGPU against 4.2 s on WebAssembly.

For native ONNX Runtime (CPU or CUDA) or WebAssembly, use the original. transcribe.py is the original's reference script, unchanged.

This copy is not made or endorsed by FluidVoice, NVIDIA or Moondream.