voice-sage95m-fp16

FP16 ONNX export of ai-forever/sage-fredt5-distilled-95m (SAGE Russian spelling correction, FRED-T5 distilled 95M) for on-device transcript polishing in the «Диктофон» voice-recorder app.

Replaces the earlier int8 export: ONNX Runtime's int8 MatMulInteger kernel crashes natively (SIGILL/SIGSEGV) at inference on some ARM phone CPUs (i8mm/dotprod path). FP16 uses ordinary MatMul kernels that don't depend on those instructions, at ~2x the int8 size but a fraction of fp32.

Files (encoder + no-KV-cache decoder, matching the app's greedy decode loop):

  • encoder_model.onnx (~98 MB)
  • decoder_model.onnx (~162 MB, no KV cache)
  • vocab.json, merges.txt - GPT-2 byte-level BPE (FRED-T5), unchanged.

Special tokens: BOS <LM>=50357, EOS </s>=2, decoder start <pad>=0. License: MIT (inherited from the base model).

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for krut42/voice-sage95m-fp16

Quantized
(4)
this model