ONNX - gliner_multi-v2.1

FP32 = model.onnx

INT8 = model_quantized.onnx

Usage

# onnxruntime-gpu for run it on GPU
pip install onnxruntime sentencepiece numpy protobuf

• src/helper.py = onnx session builder (provider selection, threads, memory options)
• src/example.py = full pipeline: word split, sentencepiece tokenization, span enumeration, inference, greedy non-overlap selection

python3 src/example.py

0.970778 | person | Linus Torvalds
0.603953 | organization | Linux
0.978727 | place | Helsinki
0.859634 | organization | Linux Foundation
0.944801 | person | 田中太郎
0.674417 | organization | 東京大学
0.953844 | place | Tokyo

Note:

• No transformers/tokenizers dependency: the tokenizer is spm.model loaded via proto with the patch
  normalizer_spec.add_dummy_prefix = False (required for the word-level alignment).
• Entity types are free text, passed in the input sequence:
  [cls] [ent] type... [ent] type... [sep] word... [eos], with [ent] = 250103, [sep] = 250104.
• words_mask marks the first subword of each word with the word number (1-based), 0 elsewhere.
• Digits are tokenized one at a time (\d|\D+), CJK characters are split one word each.
• Spans are enumerated per word up to width 12, score = sigmoid(logit), greedy selection without overlap.
• Normalize input with NFKC for consistent multilingual scores.