sinanalyuruk/trmteb-trnews-v1

trmteb-trnews

Turkish sentence-embedding model: trmteb/turkish-embedding-model fine-tuned with MultipleNegativesRankingLoss on (title, abstract) pairs from the TR-News corpus. This is a retrieval fine-tune: it improves title↔abstract retrieval; broad topical clustering is unchanged (the baseline already sits near the label-noise ceiling of the TR-News news-desk labels).

An Apple-Silicon MLX float16 build (verified at parity) is also available as trmteb-trnews-v1-mlx.

Training

• Base: trmteb/turkish-embedding-model (BERTurk, 768-d, mean pooling)
• Data: 100,000 filtered TR-News (title, abstract) pairs
• Loss: MultipleNegativesRankingLoss (in-batch negatives)
• Hyperparameters: batch 128, lr 2e-05, 1 epoch(s), bf16
• Max sequence length: 512 (the base model's capability; training truncated the short title/abstract pairs at 128 tokens, but the published model is not capped at 128)

Evaluation (held-out TR-News test + TrGLUE-STS)

Topic metrics use the full TR-News test split (n≈14k). topic kNN@1 / macro-kNN@1 / separation are held-within-noise gates, not optimisation targets.

metric	baseline	fine-tuned
title→abstract R@1	0.8635	0.8745
title→abstract MRR	0.8989	0.9093
topic kNN@1	0.7635	0.7624
topic macro-kNN@1	0.5858	0.591
topic separation	0.0276	0.0315
TrGLUE-STS spearman	0.6686	0.6677

Benchmarks (Apple M4 Max, MLX / Metal)

Head-to-head vs multilingual embedders on Turkish news (compare_embedders.py, one MLX process each). Separation = same−diff cluster cosine, which governs a threshold-based dedup cut:

metric	trnews-v1 (fp16)	base trmteb (fp32)	gemma-300m	granite-r2	qwen3-0.6B
Clustering P@1	0.989	0.993	0.943	0.975	0.939
Title→body R@1	0.983	0.983	0.967	0.967	0.933
Separation	0.655	0.661	0.170	0.120	0.345
Speed (ms/title)	0.79	0.84	1.30	1.13	5.97
Peak GPU (MB)	1713	2065	1338	1418	2015

The trmteb family holds a large separation lead — a stable ~0.35 dedup threshold vs a razor-thin, unstable ~0.80 for the multilingual models. The fp16 MLX build ≈ the fp32 base here, at lower memory and latency. (This clustering ground truth is trmteb-authored, so it is base-biased; the fine-tune's measured gain is the TR-News retrieval table above.)

Throughput (MLX/Metal vs PyTorch/MPS, full 12k-article corpus, same BERTurk arch): MLX ≈ 373 art/s, ~1.87× MPS (32 s vs 60 s), flat 2–3 GB peak — a full corpus re-embed ≈ 30 s.

Intended use & limitations

Turkish-news title↔abstract retrieval and same-event dedup. Broad topic classification, general STS, and sentiment are out-of-target — the fine-tune does not improve topical clustering over the base model; evaluate on your own task before relying on it.

License & attribution

Released under the MIT license — matching the verified upstream root (dbmdz/bert-base-turkish-uncased, MIT).

NOTICE — this is a derivative work. Attribution chain:

• Fine-tuned from trmteb/turkish-embedding-model — no license is declared upstream (as of 2026-07-03 the source model card states none); attribution retained.
• Whose base is dbmdz/bert-base-turkish-uncased (BERTurk) — MIT. Retain BERTurk's copyright and permission notice (see its model card).
• Changes (documented for provenance): fine-tuned on 100,000 TR-News (title, abstract) pairs with MultipleNegativesRankingLoss; published at max_seq_length 512.

trmteb/turkish-embedding-model itself declares no license, so the MIT terms above inherit from the BERTurk root and cover this fine-tune's contribution; confirm the intermediate model's terms with its author before commercial redistribution.

Citation — training data (TR-News)

Please cite the TR-News dataset paper:

@article{10.1007/s10579-021-09568-y,
    year = {2022},
    title = {{Abstractive text summarization and new large-scale datasets for agglutinative languages Turkish and Hungarian}},
    author = {Baykara, Batuhan and Güngör, Tunga},
    journal = {Language Resources and Evaluation},
    issn = {1574-020X},
    doi = {10.1007/s10579-021-09568-y},
    pages = {1--35}
}

TR-News license is unspecified upstream; this fine-tune cites the paper per the dataset authors' request. Confirm terms before commercial use.