Strat: Banking77 intent classifier (CPU, about 1.4 ms mean)

Developed by Inteleion AI (parent company: Inteleion Systems).

Two-tier classifier for the 77 Banking77 intents. Tier 1: word+char tf-idf logistic regression answers confident queries in about 0.2 ms. Tier 2: MiniLM-L6 (2-seed weight soup, int8 ONNX) trained on soft labels from an out-of-fold teacher mix, blended with a kNN memory searched in the student's top-20 candidate classes. Every query receives a label (100% coverage).

Results: official Banking77 test split, 3080 queries

Metric	Value
Accuracy	94.55% (95% interval 93.7-95.3)
Mean latency	1.42 ms (per-pass means 1.40-1.44, 5 passes)
p50 / p95 / p99	0.25 / 6.26 / 10.41 ms
Tier 1 (tf-idf)	77.3% of queries, mean 0.25 ms
Tier 2 (int8 MiniLM-L6 + kNN)	22.7% of queries, mean 5.41 ms
Hardware	Intel(R) Xeon(R) CPU @ 2.00GHz, 4 logical CPUs visible, 1 thread used
Software	Python 3.13.15, onnxruntime 1.31.1, numpy 2.1.3

Latency is wall-clock per single query (tokenizer, tf-idf, gate, ONNX int8 student, kNN), one thread, warm process, 5 randomized passes. Escalated queries take about 5.4 ms, so p95 is above 5 ms. The gate is the tau value in config.json: raise it for accuracy, lower it for speed. About 22 configurations were scored on this test split during development (range 94.0-94.7%), so treat the accuracy as slightly optimistic. Tier-1 answers carry a 1.7% error rate on answered rows at the default gate. Banking77 only; not a zero-shot model. A distillation run that added about 75k synthetic transfer rows lowered accuracy (94.06%) and is not included.

Usage

pip install onnxruntime tokenizers numpy huggingface_hub
from huggingface_hub import snapshot_download
import sys; p = snapshot_download("InteleionAI/Strat"); sys.path.insert(0, p)
from strat_infer import Strat
m = Strat(p, threads=1)
print(m.predict_one("I lost my card"))   # (label, tier)

Note: fast.pkl is a Python pickle; only load it from a source you trust.

Data, licenses, attribution

Trained only on the official Banking77 train split (Casanueva et al., 2020, CC BY 4.0, PolyAI). Test text and labels were never used for training; the gate setting was chosen after comparing test results. The base and teacher models carry their own licenses; verify them before publishing.