ido-pluto/SamBERT-Hebrew-Embedding-GGUF

SamBERT - Hebrew Embedding GGUF

A Q4_K_M GGUF conversion of MPA/sambert, a Hebrew sentence embedding model. All credit for the original model and its training belongs to its authors. This repository provides a format conversion and quantization; it does not introduce additional training.

SamBERT produces 768-dimensional embeddings for sentences and paragraphs, suitable for semantic search, sentence similarity, and clustering. It uses a BERT encoder with mean pooling and supports sequences up to 512 tokens.

Model file

File	Quantization	Size
sambert-Q4_K_M.gguf	Q4_K_M	138,000,992 bytes (138.00 MB)

SHA-256: d738d9403554d32f183a4f5ae1f9bf0b69aaf5b0c16df46a805017f41312ae30

Usage with llama.cpp

Download sambert-Q4_K_M.gguf and run:

llama-embedding \
  -m sambert-Q4_K_M.gguf \
  -p 'אמא הלכה לגן' \
  --pooling mean \
  --embd-normalize 2 \
  --embd-output-format json

This example returns an L2-normalized sentence embedding. Use the same model and normalization settings for both indexed documents and search queries.

Conversion and verification

• Source: MPA/sambert, revision fa7f994cc23c2ed843512369f5074d046cf5f795.
• Converted through F16 using llama.cpp converter revision f174553e1621c6a22a64e7bfcbb58d3a5c49e6fb, then quantized to Q4_K_M.
• Loaded and executed with llama.cpp build 10964 (b29c606e2) on CPU, producing finite 768-dimensional embeddings.
• On six short samples (five Hebrew, one English), the mean cosine agreement between normalized quantized embeddings and original Hugging Face embeddings was 0.98486, with a range of 0.97543–0.98879. This small conversion check is not a retrieval benchmark.

The conversion preserves vocabulary IDs and embedding rows. Eleven unreachable NUL-containing or empty continuation vocabulary entries are represented by unique unused placeholders for GGUF loader compatibility; BERT text normalization removes those control characters before tokenization.

See the original model card for training details and the authors' underlying Hebrew BERT research citation.