litoo13/leafmind-qwen3-reranker-0.6b

Qwen3-Reranker-0.6B — yes/no ONNX export for leafmind

An ONNX export of Qwen/Qwen3-Reranker-0.6B (revision e61197ed45024b0ed8a2d74b80b4d909f1255473), made for the accurate mode of leafmind's question answering: it picks the sentence of a document that answers a question.

What was changed (Apache-2.0 §4b)

The weights are the original ones, unchanged, in full precision (float32). Only the graph around them differs:

• Output: only the two scores the reranker needs — the logits of the tokens yes and no at the last position (yes_no, shape [1, 2]). The output layer keeps just those two rows instead of the whole vocabulary, so the model does far less work per text and needs less memory.
• Input: input_ids (int64, shape [1, n]) only — one text per run, no padding, no attention mask, no key/value cache.
• Why full precision: 8-bit versions of this model give different scores on x86 and ARM CPUs with ONNX Runtime (confidence differences above 0.3 on about a fifth of test questions); float32 gives the same scores on both.

Use

Score = softmax(yes_no)[0], the probability of "yes". Build the text exactly as the original model card says, encoded without adding special tokens:

<|im_start|>system
Judge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be "yes" or "no".<|im_end|>
<|im_start|>user
<Instruct>: {instruction}

<Query>: {question}

<Document>: {sentence}<|im_end|>
<|im_start|>assistant
<think>

</think>

(The last line is followed by one more empty line: the text ends with </think>\n\n.)

Files

File	SHA-256
model.onnx (graph, opset 18)	ec9d799e3a241bf06a5ceb0a1efb7d4a1645c19bd57ceed81ce89af4de2dde3b
model.onnx.data (weights, 2.38 GB)	22ab01e5d02189a8a4eac7f9da1ac027ad8f064635390da8797269f805399dda
tokenizer.json (unchanged from the original)	aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4

Keep model.onnx and model.onnx.data in the same folder.

How it was made

export.py (torch 2.14, transformers 5.17, torch.onnx.export with dynamo, opset 18). Checks: the two-row head matches the full model within 3.3e-6; the ONNX file matches PyTorch within 5e-5 (9, 74 and 339 tokens).

Licence

Apache-2.0, as the original model (see LICENSE). Qwen3-Reranker is made by the Qwen team, Alibaba Cloud.