bigscience/mt0-small — int8 ONNX
Derived from bigscience/mt0-small (Apache-2.0).
What this is
Dynamic-int8 ONNX Runtime quantization of the fp32 ONNX exports shipped in the
upstream bigscience/mt0-small model repo:
| File | Source | Size |
|---|---|---|
onnx/encoder_model_int8.onnx |
onnx/encoder_model.onnx (560.7 MB fp32) |
140.4 MB |
onnx/decoder_model_int8.onnx |
onnx/decoder_model.onnx (1073.3 MB fp32, SIMPLE 3-input) |
268.7 MB |
tokenizer.json |
unmodified byte-for-byte | 15.6 MB |
Produced with onnxruntime.quantization.quantize_dynamic(..., weight_type=QuantType.QInt8)
(onnxruntime 1.28.0). IO contract preserved: encoder
input_ids/attention_mask → last_hidden_state [1, seq, 512]; decoder
input_ids/encoder_hidden_states/encoder_attention_mask → logits
[1, seq, 250112]. No past_key_values inputs (SIMPLE graph, not merged), no
use_cache_branch input.
Model
bigscience/mt0-small is a 300M-parameter MT5ForConditionalGeneration model:
T5 encoder-decoder architecture pretrained on mC4 (101 languages) and
instruction-tuned on xP3 (46 languages incl. English, Spanish, German).
- Architecture:
MT5ForConditionalGeneration - Parameters: 300M
- Pretraining: google/mT5, mC4 — 101 languages
- Instruction tuning: bigscience/mT0, xP3 — 46 languages
- Vocab size: 250112
Usage note
This mirror exists so any Android app can download the model at runtime over public Hugging Face (no gating/authentication). It was produced for on-device int8 inference via ONNX Runtime Android 1.18.
Attribution & license
- Upstream model:
bigscience/mt0-small— Apache-2.0 - Pretraining: google/mT5 (mC4, 101-language multilingual T5)
- Instruction tuning: bigscience/mT0 (xP3)
- This quantization: Apache-2.0
The redistribution/attribution obligation is carried by this model card.