ONNX - bge-reranker-v2-m3

FP32 = model.onnx, model.onnx_data

INT8 = model_quantized.onnx

Usage

# onnxruntime-gpu for run it on GPU
pip install onnxruntime sentencepiece numpy

• src/helper.py = onnx session builder (provider selection, threads, memory options)
• src/example.py = full pipeline: sentencepiece tokenization, batching, inference, sigmoid score

python3 src/example.py

0.953583 | The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.
0.000279 | hi
0.994547 | パンダはクマ科の哺乳類で、中国の固有種である。

Note:

• No transformers/tokenizers dependency: the tokenizer is sentencepiece.bpe.model directly, with the fairseq id remap:
  (offset +1, unk = 3, bos = 0, pad = 1, eos = 2).
• Input format per pair: [bos] prompt [eos] [eos] text [eos], padded per batch to the longest sequence.
• Score = sigmoid(logit), range 0..1.
• Normalize input with NFKC for consistent multilingual scores.