OpenThai-SystemOne ONNX
This is a bundled ONNX Runtime conversion of iapp/OpenThai-SystemOne with
FP32, INT8, and INT4 versions.
The accepted structural limit is 262,144 tokens, inherited from the Qwen3.5 text backbone. PyTorch/ONNX numerical parity is validated through 65,536 tokens only.
The model requires ONNX Runtime 1.30.0.
Usage
Clone this model repository and run uv sync --frozen once to install its
locked helper package and runtime dependencies, then:
from openthai_systemone_onnx import OnnxSystemOneClient, Choice
client = OnnxSystemOneClient("imtk/OpenThai-SystemOne-ONNX")
result = client.system_one(
"บริการดีมาก",
{"sentiment": Choice(instructions="Choose sentiment", criteria={"positive": None, "negative": None})},
)
Pass precision="int8" or precision="int4" to select a quantized graph.
- INT8 dynamically quantizes activations and uses per-channel signed weights.
- INT4 uses asymmetric 32-value weight blocks with FP32 activations.
The client subclasses the upstream SystemOneClient. Formatting, public methods,
response classes, decoding, confidence, permutations, and usage accounting are
inherited unchanged.
Validation
See artifact-manifest.json for the tensor schema, operator domains, runtime
versions, source revisions, limits, and SHA-256 checksums.
Benchmark on Thai Datasets
Result Summary
OpenThaiONNX and OpenThai produced identical predictions across all 2,704 evaluated examples (Cohen's kappa and raw agreement were both 1.00). This benchmark only dips its toes in with up to 500 samples per task (204 for SIB-200). Want to help it dive deeper? Larger and more thorough evaluation contributions are very welcome!
That parity preserves OpenThai's accuracy advantage: the OpenThai models averaged 76.5% accuracy across the six selected Thai CPU task configurations, compared with 60.6% for Laya. That is a gain of 15.9 percentage points, or a 26.2% relative improvement.
OpenThaiONNX had 34.8% ± 4.4 percentage points lower CPU runtime than OpenThai This makes the ONNX version a better fit for latency-sensitive, offline, and edge applications without a dedicated accelerator, provided the device has sufficient memory for the FP32 model. _Sadly, Laya still crosses the finish line about 10× sooner than both OpenThaiONNX and OpenThai. The good news: that leaves us a delightfully large optimization playground—contributions are very welcome!_lly large optimization playground, and contributions are very welcome!
CPU
| Dataset | Model | n | Type | Accuracy | ECE | Runtime (s) |
|---|---|---|---|---|---|---|
| xnli_th | ONNX | 500 | choice | 0.8100 | 0.0246 | 407.935 |
| xnli_th | OpenThai | 500 | choice | 0.8100 | 0.0246 | 686.813 |
| xnli_th | Laya | 500 | choice | 0.6740 | 0.2347 | 39.480 |
| xnli_th | ONNX | 500 | noul | 0.8680 | 0.0247 | 407.935 |
| xnli_th | OpenThai | 500 | noul | 0.8680 | 0.0247 | 686.813 |
| xnli_th | Laya | 500 | noul | 0.8180 | 0.1511 | 39.480 |
| sib200_th | ONNX | 204 | choice | 0.7843 | 0.0816 | 126.797 |
| sib200_th | OpenThai | 204 | choice | 0.7843 | 0.0816 | 199.093 |
| sib200_th | Laya | 204 | choice | 0.7500 | 0.1572 | 7.244 |
| wongnai | ONNX | 500 | score | 0.6400 | 0.0428 | 2866.504 |
| wongnai | OpenThai | 500 | score | 0.6400 | 0.0428 | 4008.158 |
| wongnai | Laya | 500 | score | 0.2640 | 0.2493 | 167.471 |
| wisesight | ONNX | 500 | choice | 0.4960 | 0.3603 | 1397.613 |
| wisesight | OpenThai | 500 | choice | 0.4960 | 0.3603 | 2125.524 |
| wisesight | Laya | 500 | choice | 0.2800 | 0.2753 | 91.182 |
| xlam_tools | ONNX | 500 | choice | 0.9920 | 0.0072 | 448.700 |
| xlam_tools | OpenThai | 500 | choice | 0.9920 | 0.0072 | 682.976 |
| xlam_tools | Laya | 500 | choice | 0.8500 | 0.0598 | 80.015 |
GPU/MPS
| Dataset | Model | n | Type | Accuracy | ECE | Runtime (s) |
|---|---|---|---|---|---|---|
| xnli_th | ONNX | 500 | choice | ??? | ??? | ??? |
| xnli_th | OpenThai | 500 | choice | 0.8100 | 0.0246 | 165.242 |
| xnli_th | Laya | 500 | choice | 0.6700 | 0.2262 | 27.470 |
| xnli_th | ONNX | 500 | noul | ??? | ??? | ??? |
| xnli_th | OpenThai | 500 | noul | 0.8680 | 0.0247 | 165.242 |
| xnli_th | Laya | 500 | noul | 0.7640 | 0.1072 | 27.470 |
| sib200_th | OpenThai | 204 | choice | ??? | ??? | ??? |
| sib200_th | OpenThai | 204 | choice | 0.7843 | 0.0816 | 51.089 |
| sib200_th | Laya | 204 | choice | 0.6176 | 0.1412 | 5.103 |
| wongnai | OpenThai | 500 | score | ??? | ??? | ??? |
| wongnai | OpenThai | 500 | score | 0.6400 | 0.0424 | 3178.868 |
| wongnai | Laya | 500 | score | 0.0340 | 0.0173 | 124.185 |
| wisesight | OpenThai | 500 | choice | ??? | ??? | ??? |
| wisesight | OpenThai | 500 | choice | 0.4960 | 0.3603 | 2000.341 |
| wisesight | Laya | 500 | choice | 0.2500 | 0.2002 | 74.300 |
| xlam_tools | OpenThai | 500 | choice | ??? | ??? | ??? |
| xlam_tools | OpenThai | 500 | choice | 0.9920 | 0.0072 | 609.601 |
| xlam_tools | Laya | 500 | choice | 0.8500 | 0.0598 | 45.588 |
OpenThaiONNX vs OpenThai Agreement (Cohen's kappa)
| Task | n | Kappa |
|---|---|---|
| Overall | 2704 | 1.000000 |
| sib200_th / choice | 204 | 1.000000 |
| wisesight / choice | 500 | 1.000000 |
| wongnai / score | 500 | 1.000000 |
| xlam_tools / choice | 500 | 1.000000 |
| xnli_th / noul | 500 | 1.000000 |
| xnli_th / choice | 500 | 1.000000 |
Limitations
- Quantized results are approximate and should be evaluated with the included held-out benchmark for the target workload.
- The model performs complete-request inference; generation caches are deliberately internal and reset for every call.
- 262,144 tokens is an accepted graph boundary, not a performance guarantee.