GoktugD/DUSUNEN-Pusula-118M-v1

DUSUNEN Pusula 118M v1

A compact Turkish retriever with six selectable vector sizes. Pusula is a 117.7M-parameter Sentence Transformer trained with Matryoshka Representation Learning on 50,000 Turkish hard-negative triplets. It emits 384-dimensional embeddings by default and can be truncated to 256, 192, 128, 64, or 32 dimensions without storing another model.

Measured dimension sweep

All results use the same 2,000-example held-out hard-negative split and cosine similarity. The baseline is the public Pusula v0 checkpoint evaluated with the same code and text format.

Dimensions	Base accuracy	Pusula v1 accuracy	Gain	Float32 bytes/vector	Index reduction
384	92.00%	91.95%	-0.05 pp	1,536	0.0%
256	91.60%	91.70%	+0.10 pp	1,024	33.3%
192	90.70%	90.75%	+0.05 pp	768	50.0%
128	90.05%	89.85%	-0.20 pp	512	66.7%
64	88.70%	88.90%	+0.20 pp	256	83.3%
32	86.40%	86.40%	+0.00 pp	128	91.7%

These are in-domain held-out triplet results, not a universal or state-of-the-art claim. Machine-readable evidence is included in dimension-sweep.json.

Turkish retrieval benchmark

The five-task score below was measured with MTEB 2.18.16, normalized 384-dimensional embeddings, bfloat16 inference, and the plain-text input format. Dataset revisions and task metadata are recorded in mteb-five-task.json.

Task	Main score
TurHistQuadRetrieval	0.25543
XQuADRetrieval	0.81568
WebFAQRetrieval	0.46310
MKQARetrieval	0.04860
BelebeleRetrieval	0.82598
Macro average	0.481758

Scores across tasks with different metrics should be read as a compact release regression panel, not as a single universal quality measure.

Dynamic-int8 ONNX

The repository includes onnx/model_quantized.onnx (118.3 MB), exported for AVX2 CPU inference. Across the included validation probes, PyTorch-to-ONNX mean cosine agreement was 0.989893, minimum agreement was 0.979842, and example-query top-1 agreement was 100.0%. See onnx-validation.json for machine-readable details.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "GoktugD/DUSUNEN-Pusula-118M-v1",
    truncate_dim=128,
)

queries = ["Türkiye'nin başkenti neresidir?"]
passages = [
    "Türkiye'nin başkenti Ankara'dır.",
    "İstanbul Türkiye'nin en kalabalık şehridir.",
]
q = model.encode(queries, normalize_embeddings=True)
p = model.encode(passages, normalize_embeddings=True)
print(q @ p.T)

Pusula uses plain text; do not add E5-style prefixes. Apply the same truncate_dim to query and document embeddings.

Training

• Base: GoktugD/DUSUNEN-Pusula-118M-v0
• Data: 50,000 train / 2,000 held-out Turkish hard-negative triplets
• Loss: cached multiple-negatives ranking loss wrapped by Matryoshka loss
• Dimensions: 384, 256, 192, 128, 64, 32
• Effective batch size: 64; one epoch; bfloat16; seed 3407
• Hardware: NVIDIA GeForce RTX 3090 24 GB
• Training time: 1383.3 seconds
• Peak memory allocated by training: 1.082 GiB

Exact configuration, environment and evaluation scripts are included in this repository.

Limitations

• The dimension sweep is an in-domain held-out evaluation; the separate MTEB panel is broader but is not a substitute for application-specific testing.
• Very small dimensions trade ranking quality for index size.
• The underlying training corpus can contain noisy or semantically overlapping negatives.
• Similarity thresholds require application-specific calibration.

Integrity

model.safetensors SHA-256:

1ac5c7f6c40598ae744b82373eb6f5a37d6569402ebaf5dada12b3e56553d81a

Developed and released by Göktuğ Düşünen.