cyberntx/static-retrieval-384-v2
Static (sum-pooled token table) retrieval embedding model, 384 dimensions,
distilled from microsoft/harrier-oss-v1-0.6b via contrastive + teacher-
regression training over a diverse ~8.95M-row corpus (web crawl, StackExchange,
Wikipedia, CC-NEWS-ES, PMC, OpenAlex).
Status: early checkpoint, still training. More data and further training iterations are in progress.
Benchmarks
Compared against the previous checkpoint of this model (v1, trained on a
500K-row web-crawl-only corpus) and against
amgix/static-retrieval-multilingual-69m-v1,
a similarly-sized static retrieval model distilled from
ibm-granite/granite-embedding-97m-multilingual-r2.
MTEB Retrieval (NFCorpus, SciFact, ArguAna)
General out-of-domain retrieval tasks (biomedical, scientific claim verification, argument retrieval) -- none of these domains were specifically targeted by training. Compared against real public models in the same static/tiny-embedding class, not just one competitor:
| Model | Dim | NFCorpus | SciFact | ArguAna | Average |
|---|---|---|---|---|---|
| potion-retrieval-32M | 512 | 0.3070 | 0.6378 | 0.4491 | 0.4646 |
| AMGIX-69M | 384 | 0.2779 | 0.5742 | 0.4163 | 0.4228 |
| v2 (this model) | 384 | 0.2280 | 0.5563 | 0.4216 | 0.4020 |
| static-similarity-mrl-multilingual | 1024 | 0.2276 | 0.4959 | 0.4410 | 0.3882 |
| v1 (this project, 500K rows) | 384 | 0.1796 | 0.4692 | 0.3411 | 0.3300 |
v2 improves ~22% over v1 on average, beats AMGIX outright on ArguAna, and beats static-similarity-mrl-multilingual on average despite that model being 1024-dim (2.7x the size). potion-retrieval-32M remains the strongest model in this class overall and is the real bar for the next iteration.
Internal web-crawl rerank diagnostic
A 24-query hard-negative rerank diagnostic over short web-crawl title+lead excerpts -- closer to this model's original training domain than the MTEB tasks above.
| Model | top1/24 | top2/24 | top10/24 | top25/24 | MRR |
|---|---|---|---|---|---|
| v1 (500K rows) | 18 | 19 | 21 | 23 | 0.796 |
| v2 (this model) | 19 | 19 | 20 | 23 | 0.806 |
| AMGIX-69M | 21 | 21 | 22 | 24 | 0.886 |
On this domain-specific diagnostic v2 is roughly flat versus v1 (small MRR gain, small top10 regression) and still clearly behind AMGIX. The broader, more diverse training corpus that drove the MTEB gains above did not carry over to this narrower in-domain task -- an open problem for the next training iteration.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("cyberntx/static-retrieval-384-v2")
embeddings = model.encode(["your text here"])