Ozankj/MoganBERT-Embed-ONNX

MoganBERT-Embed ONNX (Unofficial)

Unofficial ONNX mirror of moganai/MoganBERT-Embed. The FP16 export preserves the original weights; model_quantized.onnx is an int8 quantized derivative intended for compact inference.

• Original model: https://huggingface.co/moganai/MoganBERT-Embed
• Paper: https://huggingface.co/papers/2608.25768
• Original authors / copyright: MoganAI (Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk Gozay)
• License: Apache License 2.0 (see LICENSE and NOTICE)
• ONNX conversion by: Ozankj

The original mean pooling and L2 normalization are retained. FP32 and FP16 exports preserve the original weights (FP16 values are represented in half precision). onnx/model_quantized.onnx stores quantized weights and requires a compatible ONNX Runtime build. See QUANTIZATION.md for custom quantization notes.

Files

File	Format	Size	Notes
onnx/model.onnx	FP32	~568 MB	Exact export; outputs match PyTorch
onnx/model_fp16.onnx	FP16	~298 MB	Recommended for compatibility; negligible output difference
onnx/model_quantized.onnx	Mixed int8 / custom graph	~154 MB	Aggressive compact custom int8 export; requires MatMulNBits support

Outputs: token_embeddings (batch, seq_len, 768) and sentence_embedding (batch, 768). The latter is mean-pooled and L2-normalized.

Usage

from sentence_transformers import SentenceTransformer

# FP16 (recommended for compatibility)
model = SentenceTransformer(
    "Ozankj/MoganBERT-Embed-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_fp16.onnx"},
)

# Compact quantized variant; requires MatMulNBits support
quantized_model = SentenceTransformer(
    "Ozankj/MoganBERT-Embed-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_quantized.onnx"},
)

sentences = ["Türkiye'nin başkenti Ankara'dır.", "Bugün hava çok güzel."]
embeddings = model.encode(sentences)

For direct ONNX Runtime use, load onnx/model_fp16.onnx and read the sentence_embedding output.

For direct ONNX Runtime use of the quantized model, load onnx/model_quantized.onnx and read sentence_embedding. It is a custom mixed int8 graph, not a standard QDQ / dynamic int8 export. It needs an ONNX Runtime build and execution provider that support MatMulNBits; embedding weights are int8 and dequantized in-graph with Cast / Mul. Do not assume compatibility with web/mobile runtimes without testing.

Accuracy

• FP32 ONNX vs PyTorch: cosine similarity 1.0 (max abs diff ~1e-05)
• FP16 ONNX vs PyTorch: cosine ~1.0 (max abs diff 8.3e-04)
• Quantized ONNX vs FP32 ONNX: near-identical on our representative smoke test of 10 Turkish sentences (average cosine 0.9996, minimum 0.9993); for a 512-token long example (average/minimum cosine 0.9994). Maximum absolute pairwise-similarity change was 0.013. These are limited smoke tests, not a full MTEB benchmark, and runtime parity is not guaranteed.
• The quantized graph requires ONNX Runtime MatMulNBits support. Use FP16 for maximum compatibility.
• See QUANTIZATION.md for the full custom quantization, validation, attribution and reproduction notes.

Not affiliated

This repository is not affiliated with, endorsed by, or maintained by MoganAI. For the official model, weights and benchmarks, use the original link above.