nchapman/jina-code-embeddings-0.5b-onnx

jina-code-embeddings-0.5b → ONNX (Transformers.js-ready)

ONNX conversion of jinaai/jina-code-embeddings-0.5b (Qwen2.5-Coder-0.5B finetune, 494M params), laid out like onnx-community/Qwen3-Embedding-0.6B-ONNX: weights under onnx/, tokenizer + config at root, transformers.js compatible.

• License: CC-BY-NC-4.0 (inherited from source — attribution required, non-commercial only).
• Export: optimum feature-extraction, opset 17, eager attention, no ORT graph optimization (unsupported for qwen2).
• Pooling: last-token is done outside the model graph; pass pooling: "last_token" (JS) or pool manually (Python, below).
• Files: onnx/model.onnx (fp32, 1.98 GB), onnx/model_fp16.onnx (944 MB).

Usage (Transformers.js)

import { pipeline, matmul } from "@huggingface/transformers";

const extractor = await pipeline(
  "feature-extraction",
  "nchapman/jina-code-embeddings-0.5b-onnx",
  { dtype: "fp32" }, // or "fp16"
);

const queries = [
  "Find the most relevant code snippet given the following query:\nprint hello world in python",
];
const documents = ["Candidate code snippet:\nprint('Hello World!')"];

const output = await extractor([...queries, ...documents], {
  pooling: "last_token",
  normalize: true,
});
const scores = await matmul(
  output.slice([0, queries.length]),
  output.slice([queries.length, null]).transpose(1, 0),
);
console.log(scores.tolist());

Usage (Python, onnxruntime)

import numpy as np, onnxruntime as ort, torch
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("nchapman/jina-code-embeddings-0.5b-onnx")
sess = ort.InferenceSession("onnx/model.onnx", providers=["CPUExecutionProvider"])

texts = [
    "Find the most relevant code snippet given the following query:\nprint hello world in python",
    "Candidate code snippet:\nprint('Hello World!')",
]
batch = tok(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
token_emb, sent_emb = sess.run(None, {
    "input_ids": batch["input_ids"].numpy().astype(np.int64),
    "attention_mask": batch["attention_mask"].numpy().astype(np.int64),
})

mask = batch["attention_mask"]
if (mask[:, -1].sum() == mask.shape[0]):
    emb = torch.from_numpy(token_emb)[:, -1]
else:
    seq_lens = mask.sum(dim=1) - 1
    emb = torch.from_numpy(token_emb)[torch.arange(len(texts)), seq_lens]
# or use sent_emb directly (sentence-transformers pooling output)

Parity (2026-10-01, model-card example, torch fp32 reference [[0.8168, 0.1236],[0.1204, 0.5525]])

variant	max cosine-score drift	verdict
onnx/model.onnx (fp32)	0.000000	parity OK
onnx/model_fp16.onnx	0.001650	ships (ranking preserved)

Why no q8?

Dynamic int8 quantization was tried (ORT quantize_dynamic, QInt8, per-channel + reduce-range, with and without quant_pre_process shape inference) and scored drift ≈ 0.145 — two orders of magnitude over the 1e-3 bar. This GQA-decoder + RMSNorm + last-token-pooling architecture is far more quant-sensitive than BERT-style encoders where int8 dynamic works. Per the sentence-transformers team's guidance, the recommended efficiency path for embeddings is embedding quantization (quantize output vectors to int8/binary at retrieval time), not model-weight quantization. A q8/int8 variant may return via static quantization with calibration data + a broader retrieval eval — not a 4-text smoke test.