N03m4/pulpie-orange-small-onnx

pulpie-orange-small (ONNX int8)

Quantized ONNX export of pulpie-small (EuroBERT token classifier).

• Graph: 2 inputs (input_ids, attention_mask, int64 [batch, seq]) → logits [batch, seq, 2]. position_ids = arange(seq) is computed inside the graph (matches the EuroBERT default).
• Recipe: fp32 torch.onnx export (opset 17, dynamic batch ≤ 64 / seq ≤ 8192) → token-embedding cast bf16 → int8 block-32 weight-only MatMuls (MatMulNBits, classifier head kept fp32).
• Parity: fp32 ONNX vs torch max|d| ~ 8e-06; int8 argmax agreement 100% (short) / 99.77% (long).

from optimum.onnxruntime import ORTModelForTokenClassification
from transformers import AutoTokenizer
m = ORTModelForTokenClassification.from_pretrained("REPO_ID", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("REPO_ID", trust_remote_code=True)
out = m(**tok("some page html", return_tensors="pt"))  # out.logits