smarttasks/bge-small-en-v1.5-GGUF

Pound-for-pound efficiency

At 33M parameters (~20 MB at Q4_K_M), this is one of the smallest embedders available — yet on our standardized SciFact dense-retrieval benchmark it scores 0.7154 nDCG@10, the strongest accuracy-per-parameter in the SmartTasks embedding line and near the top absolute score at a fraction of the size. If you are constrained by VRAM, latency, or cost, this is the efficiency pick: near-large-model retrieval quality in a file small enough to run comfortably on CPU or edge hardware.

How it compares (SmartTasks embedding line)

All scores measured the same way: SciFact, dense-retrieval mode, nDCG@10, on the Q8_0 quant served via llama.cpp. This is one task on one corpus — a useful apples-to-apples signal within this line, not a universal ranking.

Model	Params	~Q4 size	SciFact (dense)
bge-small-en-v1.5 (this)	33M	~20 MB	0.7154
all-MiniLM-L6-v2	23M	~15 MB	0.6507
nomic-embed-text-v1.5	137M	~86 MB	0.7056
mxbai-embed-large-v1	335M	~206 MB	0.7388
bge-m3	568M	~380 MB	0.6458 *

* bge-m3 caveat: 0.6458 is its dense-only score. bge-m3 also supports sparse + multi-vector (ColBERT) retrieval; in hybrid mode it scores substantially higher (~0.86 on some setups). It is a stronger and more capable model than the dense number alone suggests — it simply isn't optimized for dense-only use, which is what llama.cpp serves. Do not read this table as ""bge-small beats bge-m3"" in general; it doesn't.

Usage note

bge-small-en-v1.5 (v1.5) needs no query instruction prefix for general similarity/retrieval — use text directly. For pure passage retrieval you may optionally prepend an instruction to queries (not documents), but it is not required. Context window: 512 tokens.

bge-small-en-v1.5 — Embedding GGUF (quantization-verified)

Quantized embedding model in GGUF, served in --embedding mode via llama.cpp. This is an encoder — it outputs vectors, not text. It is validated for retrieval quality and quantization fidelity, not chat behavior.

Files

• bge-small-en-v1.5-Q4_K_M.gguf (29.1 MB)
• bge-small-en-v1.5-Q5_K_M.gguf (30.4 MB)
• bge-small-en-v1.5-Q8_0.gguf (36.7 MB)

Quantization drift (vs f16)

Mean cosine similarity of embeddings vs the f16 baseline. 1.0 = identical.

Quant	Mean cosine	Min cosine	Verdict
Q4_K_M	0.99689	0.99594	excellent (>0.99)
Q5_K_M	0.99764	0.99619	excellent (>0.99)
Q8_0	0.99985	0.99977	excellent (>0.99)

Per-domain fidelity at Q4_K_M (which content types the quant preserves best):

Domain	Mean cosine	Min
finance	0.99641	0.99616
legal	0.99644	0.99594
short_queries	0.99644	0.99598
everyday	0.99691	0.99658
long_form	0.99701	0.99693
code	0.99725	0.99693
medical	0.99725	0.99688
science	0.99747	0.99712

Retrieval sanity (lightweight)

Built-in 12-query retrieval check (no external corpus): top-1 accuracy 1.0, MRR 1.0. healthy (top-1 >= 0.9)

Retrieval (MTEB)

Standardized MTEB retrieval scores (main metric, usually nDCG@10 — higher is better). These are comparable across models on the MTEB leaderboard.

Task	Score
SciFact	0.7154

Metric: main_score (retrieval tasks: nDCG@10). Measured on the Q8_0 quant served via llama.cpp.

Dense-retrieval mode. These scores are for standard single-vector dense retrieval (what llama.cpp serves). Models like BGE-M3 that also support sparse/multi-vector (ColBERT) modes score higher in hybrid setups — that capability isn't exercised here, so compare this number against other models' dense scores, not hybrid ones.

What this is NOT

This card carries no safety, red-team, or viewpoint scores: those do not apply to an embedding model. For chat-model governance cards, see the SmartTasks text-LLM line.