patronus-studio/panther-read-intent-classifier-edge

Model Card for Panther Read Intent Classifier Edge

Quantized ONNX builds of Panther Read Intent Classifier for local / edge inference.

This repository contains the size- and memory-optimized ONNX exports of the FP32 model. For intended uses, class definitions, training data, benchmarks, and limitations, see the main model card.

Variants

Variant	Path	Quantization
FP16	onnx/fp16/model.onnx	FP16 weights
INT8	onnx/int8/model.onnx	Post-training dynamic INT8 (MatMul/Gemm)
INT8 + INT4 embeddings	onnx/int8_int4_embeddings/model.onnx	Dynamic INT8 plus 4-bit weight-only token embeddings

The exact quantization parameters are recorded in onnx/quantization_manifest.json.

Benchmark

Quantization parity on the unified_v3 held-out test split (n = 1880; max_length 128, CPU):

Variant	Accuracy	F1 (macro)	Δ F1 vs FP32
FP32 reference (main repo)	0.8984	0.8992	—
FP16	0.8957	0.8966	-0.0026
INT8	0.8941	0.8962	-0.0030
INT8 + INT4 embeddings	0.8931	0.8940	-0.0052

Measured CPU latency (batch 64): ~18.98 ms/text for the INT8+INT4 build.

Full details in metrics/quant_bench.json.

Usage

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer

model_id = "patronus-studio/panther-read-intent-classifier-edge"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSequenceClassification.from_pretrained(model_id, subfolder="onnx/int8_int4_embeddings")

inputs = tokenizer("...", return_tensors="pt")
logits = model(**inputs).logits

Post-process the logits as described in the main model card (softmax over the logits, argmax).

License

Released under the Apache License 2.0; the full text is included as LICENSE. Derived from Panther Read Intent Classifier and ultimately from jhu-clsp/mmBERT-small (MIT License); upstream notices are retained.

Patronus Ark

This model is built to run inside Patronus Ark, Patronus' open-source on-device AI-security scanning library (L1 native rules → L2 NTDB cascade → L3 transformer). Ark is not publicly released yet — a repository link will be added here at launch.

---

🛡️ Patronus Protect

Brought to you by Patronus Protect — a local AI firewall that secures every AI interaction, including prompts, tools and documents, before it reaches your models.

Try it for free at patronus.studio.