ONNX + CTranslate2 (INT8) edge builds, with a measured RTF benchmark
Thank you for releasing Breeze-ASR-26 under Apache-2.0 — it is the first Taigi ASR we could actually deploy on real hardware.
We converted it to two edge runtimes, quantized both to INT8, and measured them on real multi-speaker Mandarin meeting audio (spontaneous speech with crosstalk, not synthetic clips), processed in full 30 s windows on CPU with 4 threads:
| Runtime | Size | RTF (CPU, 4 threads) | Real-time? |
|---|---|---|---|
| CTranslate2 / faster-whisper | 1.5 GB | 0.18 – 0.23 | yes |
| ONNX / sherpa-onnx | 1.7 GB | 1.17 – 1.76 | no (but runs on Android / iOS / WASM) |
Models: https://huggingface.co/weemed/Breeze-ASR-26-edge (Apache-2.0, all credit for the model itself to MediaTek Research — we contribute only format conversion, quantization and measurement.)
Three findings that may be useful to other users of this model:
Mandarin does not regress. Despite the Taigi fine-tune, transcription of pure Mandarin meetings is fully usable in our tests, so a dual-model router (ASR-25 for Mandarin + ASR-26 for Taigi) turned out to be unnecessary.
Benchmarks on short clips are misleading. Whisper pads every input to a 30 s window, so a 2.5 s clip costs nearly as much encoder time as a 30 s one. We measured RTF 0.88 on 2.5 s clips and RTF 0.17 on a repetitive 30 s clip — both flattering and both wrong. Only dense, full-window speech gives an honest number.
Thread count barely helps. 4 / 8 / 16 threads gave RTF 1.28 / 1.52 / 1.39 for ONNX — the autoregressive decoder is the bottleneck, not parallelism. CTranslate2's advantage comes from its KV-cache and fused kernels.
We also verified the Mandarin-character output behaviour documented in your card, against ground truth:
| Ground truth (Taigi) | Output |
|---|---|
| 我沒咧驚 (guá bô leh kiann) | 我沒在怕 |
| 烏白試 (oo-pe̍h tshì) | 黑白試 |
Semantics correct, characters differ — worth highlighting for anyone expecting verbatim Taigi transcripts.
One open gap on our side: we could not find a labelled code-switched (Mandarin↔Taigi within one utterance) sample to evaluate against. The benchmark in the paper ships parallel monolingual pairs. If such a set exists (even a handful of clips), we would happily measure and report it.
Feel free to link these builds from the model card, or let us know if you would prefer them hosted differently.