strizh-ru-retriever

A tiny (24.4M), no-prefix dense retriever for Russian. A 4-layer sentence encoder layer-pruned and re-trained from deepvk/RuModernBERT-small. It produces a single 384-dimensional embedding per text, needs no query/document prefixes, and its architecture accepts up to 8192 tokens (the contrastive training used 256-token sequences). Built as a fast first-stage retriever for self-hosted Russian RAG where the encoder shares one GPU with an LLM and a reranker (e.g. an AMD Strix Halo box) — a small, fast embedder returns GPU time to generation.

Honest positioning. strizh is a specialist, not a quality leader. It beats rubert-tiny2 on Russian retrieval (0.75 vs 0.63 on the clean subset below) and on Strix Halo / Vulkan throughput — but rubert-tiny2 is faster on CPU. Larger models beat strizh on quality: its sibling deepvk/USER2-small (+10M params, prefix-based), multilingual-e5-small and BAAI/bge-m3 are ahead on Russian, English and cross-lingual retrieval. Pick strizh when it is RU-first, when no prefixes must hold, and when first-stage speed on a shared GPU matters — behind hybrid search and a reranker. See When to use.

Specification

Base model	deepvk/RuModernBERT-small
Parameters	24.4M
Layers	4
Hidden dim	384
Vocab	50 368 (Russian + English)
Context length	8192 (architecture; contrastive training used 256)
Pooling	mean
Output	single dense vector, L2-normalized (cosine)
Retrieval mode	dense (single-vector) only — see Retrieval design

Usage

Drop-in — no prefixes, no instructions. Queries and documents are encoded the same way.

sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("AGmind/strizh-ru-retriever")

query = "как включить mean-пулинг у эмбеддера"
docs = [
    "Для llama-server задайте --pooling mean, чтобы получить усреднённый эмбеддинг.",
    "Reranker сортирует кандидатов после первичного поиска.",
]

q = model.encode(query, normalize_embeddings=True)
d = model.encode(docs, normalize_embeddings=True)
print((d @ q))   # cosine similarities

Transformers

import torch, torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

tok = AutoTokenizer.from_pretrained("AGmind/strizh-ru-retriever")
model = AutoModel.from_pretrained("AGmind/strizh-ru-retriever")

def encode(texts):
    batch = tok(texts, padding=True, truncation=True, return_tensors="pt")
    with torch.no_grad():
        out = model(**batch).last_hidden_state
    mask = batch["attention_mask"].unsqueeze(-1).float()
    emb = (out * mask).sum(1) / mask.sum(1)      # mean pooling
    return F.normalize(emb, p=2, dim=1)

llama.cpp (GGUF, AMD / Vulkan / CPU)

llama-server -m strizh-ru-retriever.Q8_0.gguf \
  --embeddings --pooling mean -ngl 999 -c 8192 -b 4096 -ub 4096

GGUF weights: see AGmind/strizh-ru-retriever-GGUF. --pooling mean is required — the default cls collapses recall for this model.

Context caveat: -c is the total server context, divided across --parallel slots, and an embedding input must also fit into one ubatch (-ub). With -c 8192 -np 8 each slot gets only 1024 tokens and longer inputs return HTTP 400. To actually accept documents up to 8192 tokens with 8 slots use -c 65536 -b 8192 -ub 8192 -np 8 — the 4-layer KV cache is tiny, so this is cheap (verified: ~5000-token inputs embed fine).

Retrieval design

strizh implements one retrieval technology: dense, single-vector semantic search. This is deliberate. In a modern RAG stack the other retrieval modes are supplied by the surrounding components, so the embedder does not need to carry them:

mode	strizh	typically provided by
dense (semantic)	✅	this model
sparse / lexical (BM25)	—	the vector DB (e.g. Milvus native BM25 / hybrid search)
precision re-ranking	—	a cross-encoder reranker (e.g. bge-reranker-v2-m3)

Contrast bge-m3, which bundles all three modes into one 568M model. strizh takes the opposite bet: a tiny dense specialist that leaves sparse and re-ranking to the stack. Use it as the first-stage candidate generator in a hybrid + rerank pipeline.

Training

Derived from deepvk/RuModernBERT-small (which is itself Russian+English, Apache-2.0):

• Warm-start layer pruning. The 12-layer donor is pruned to 4 layers ([0, 5, 9, 11]), keeping the shared 50 368-token embedding table, then re-trained with a contrastive objective — so the encoder is small in depth, not in vocabulary. (This is layer selection + warm-start + contrastive training, not classic online distillation.)
• Contrastive objective. Multiple-Negatives Ranking Loss (in-batch negatives) plus hard negatives mined with bge-m3 from a mixed RU/EN candidate pool (the miner does not enforce that a negative is cross-lingual).
• Data. Russian, English and mixed (Russian prose + code / English identifiers) pairs — MIRACL (Apache-2.0), opus-100, and ru_stackoverflow (CC BY-SA). Check each source's license for your use case; the base model is Apache-2.0.

Evaluation

RU retrieval uses our reproducible MIRACL-ru dev harness (first 1000 queries, dev passage corpus, 512-token cap, each model in its native pooling/prefix mode). strizh's donor training saw some MIRACL passages, so we report the passage-exposure-controlled subset — the 758 of 1000 queries whose gold passages never appeared as training positives — as the fair number. (On the full 1000-query set strizh scores 0.80; the filtered subset is also simply harder — every model scores lower on it — so the delta is not pure leakage, and the full-set score is best read as potentially optimistic due to that exposure.) EN = NanoBEIR mean nDCG@10 (via MTEB, prompt handling for prefix baselines relied on MTEB's model-metadata prompts — treat baseline EN numbers as approximate), not recall; cross-lingual = opus-100 translation-pair retrieval: ru↔en recall@10 (mean of both directions). Throughput = AMD Strix Halo, llama.cpp Vulkan, Q8_0, warmed, concurrency 16; across reproduced runs this point ranges 3614-4450 req/s, so read it as "about four thousand". Columns use different metrics — do not average across them. Bold = best in column.

model	params	RU recall@10 (filtered)	EN nDCG@10	opus-100 pair R@10	Strix short-query req/s	drop-in
bge-m3	568M	0.83	0.60	0.96	435	✅
mE5-small	118M	0.83	0.55	0.94	1837	—
USER2-small	34M	0.82	0.53	0.95	2635	—
strizh-ru-retriever	24M	0.75	0.34	0.76	~4000	✅
rubert-tiny2	29M	0.63	0.17	not measured	2362	✅

Read honestly: strizh beats rubert-tiny2 on Russian (0.75 vs 0.63) and is the fastest on the tested Vulkan stack — but it sits below USER2-small, mE5-small and bge-m3 on retrieval quality, and its English (0.34) and cross-lingual (0.76) axes are weak. A ru_stackoverflow "mixed" score exists too, but that eval overlaps the training data, so it is a training-diagnostic rather than a benchmark and is omitted here.

Real-world validation (dogfood RAG)

On a separate corpus from a different domain (the project's own repository — not part of strizh's retrieval training data; 1699 chunks of Russian docs + Python + config, 163 synthetic single-gold questions, bge-reranker-v2-m3 after retrieval), strizh reproduced the same picture end-to-end: it clearly beat rubert-tiny2 (recall@10 0.62 vs 0.53 after rerank) and stayed competitive with USER2-small on Russian strata, but cross-lingual retrieval stayed at 0.35 before rerank and 0.38 after it — if the right chunk is not in the candidate set, no reranker recovers it.

When to use

Use strizh when all of these hold:

• the corpus is Russian-dominant;
• you cannot (or do not want to) inject query/document prefixes — frozen RAG stacks, OpenAI-embeddings-shaped clients;
• first-stage throughput/latency on a shared GPU is binding — the encoder shares one GPU with an LLM and a reranker, so a fast small embedder returns compute to generation;
• it sits behind hybrid search and/or a reranker (as a first-stage retriever), not as the sole ranking signal.

Do not use strizh for:

• cross-lingual retrieval (English query → Russian document, or vice versa) — 0.76;
• English-heavy or mixed-language / code-symbol retrieval — 0.34 English;
• CPU-only inference — rubert-tiny2 is faster there (357 vs 265 emb/s) and strizh has no ONNX path yet;
• pure-vector stores with no lexical/rerank backstop when accuracy dominates over size.

If you can add prefixes, prefer USER2-small; if you have GPU/latency headroom, bge-m3 (also drop-in) is higher quality across the board.

GGUF quantizations

AGmind/strizh-ru-retriever-GGUF:

file	size	notes
strizh-ru-retriever.F16.gguf	52 MB	reference precision
strizh-ru-retriever.Q8_0.gguf	29 MB	recommended
strizh-ru-retriever.Q5_0.gguf	27 MB	lower bits, barely smaller
strizh-ru-retriever.Q4_0.gguf	26 MB	not recommended (see below)

Just use Q8_0. The 50 368-token embedding table dominates the file and stays near 8-bit regardless, so Q5_0/Q4_0 save only ~2–3 MB over Q8_0 while losing quality — there is no reason to go below Q8_0 for this model. K-quants (Q4_K / Q6_K) are unavailable: hidden 384 / intermediate 576 are not multiples of the K-quant super-block (256), so llama-quantize silently falls back to legacy quants. Always serve with --pooling mean.

vLLM (NVIDIA)

The same checkpoint serves under vLLM (NVIDIA) as well as llama.cpp / Vulkan (AMD) and sentence-transformers (CPU/GPU). To serve under vLLM:

vllm serve AGmind/strizh-ru-retriever \
  --pooler-config '{"pooling_type":"MEAN"}' --max-model-len 8192

The repo ships the sentence-transformers layout (modules.json + pooling + normalize) and tokenizer_class = PreTrainedTokenizerFast so vLLM loads it as a MEAN-pooled embedding model. Pass truncate_prompt_tokens in requests if your inputs can exceed --max-model-len.

Limitations

• Cross-lingual and English retrieval are weak (0.76 / 0.34). This is a RU-first model with residual English support, not an equal bilingual retriever.
• Dense-only: it does not do lexical or multi-vector retrieval — pair it with hybrid search and a reranker.
• Quality is below every larger Russian retriever; the value is speed on the target llama.cpp/Vulkan stack and drop-in (no-prefix) simplicity, not accuracy.

License

Model weights: Apache-2.0 (base deepvk/RuModernBERT-small is Apache-2.0). Note the training data mixes licenses — ru_stackoverflow is CC BY-SA, MIRACL is Apache-2.0 — so review data provenance for your own use case rather than assuming a single permissive license.

Citation

@misc{strizh_ru_retriever_2026,
  title  = {strizh-ru-retriever: a tiny drop-in dense retriever for Russian RAG},
  author = {AGmind},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/AGmind/strizh-ru-retriever}}
}