furkanbekmezci/turklink-reranker

TurkLink Turkish cross-encoder

BERTurk scalar compatibility logits trained with listwise hard-negative ranking. Use scores across candidates, not a single-pair probability.

This is one component of frcturus/turklink-el. The results below refer to that complete pipeline, not this component alone.

Install the library from the wheel in the full bundle, then:

from turklink.models import CrossEncoder
from turklink.schema import Mention
from turklink.text import Morphology, mention_query, entity_text
model = CrossEncoder("frcturus/turklink-reranker", max_length=192, device="cpu")
text = "Apple İstanbul'da yeni mağazasını açtı."
query = mention_query(text, Mention(0, 5, "Apple"), Morphology("apostrophe"))
candidate = {"label": "Apple Inc.", "aliases": ["Apple"],
             "description": "Çok uluslu Amerikan teknoloji şirketi"}
logit = model.score([query], [entity_text(candidate)])[0]
# Compare logits over retrieved candidates; the logit is not confidence.

Measured pipeline results

Dataset	Mentions	Accuracy / micro-F1	R@1	R@5	R@10	R@32	Catalog coverage
mewsli-9-tr	5811	0.8226	0.7567	0.8845	0.9029	0.9248	0.9955
mewsli-x-tr-dev	262	0.9008	0.7863	0.9389	0.9542	0.9656	0.9962
mewsli-x-tr-test	1215	0.8593	0.7778	0.8963	0.9185	0.9342	0.9951
test	2000	0.8020	0.7500	0.8695	0.8900	0.9050	0.9985
validation	2000	0.8335	0.7725	0.8900	0.9105	0.9335	0.9980

These are measured results of this sampled pilot, not the TurkLink paper's results. Gold spans are supplied for the table. Accuracy equals micro-F1 because each mention has one gold and one predicted Q-ID. The catalog has 3,958,456 Q-IDs; missing gold entities count as misses. Candidate retrieval never inserts the correct answer.

Mewsli results, when present, are Turkish supervised domain transfer against this catalog, not the original zero-shot XTREME-R protocol/candidate descriptions. TurkLink end-to-end metrics are sparse-hyperlink proxies and are in the JSON reports; they must not be presented as exhaustive NER accuracy.

Raw-text end-to-end proxy

Dataset	Windows	EL precision	EL recall	EL micro-F1	Mention micro-F1
test	300	0.2286	0.6429	0.3372	0.4052
validation	300	0.2073	0.6559	0.3150	0.3669

These exact-span measurements include mention detection. Unannotated but correct mentions are counted as false positives by the sparse hyperlink reference.

Data and reproducibility

TurkLink, Akdaş & Tantuğ (2026), DOI 10.1016/j.procs.2026.01.041. Dataset revision: c8d7fe7bdd0ae934268d30ef64f6444940bdf6dc. Official split membership is preserved; sampling is by document. The catalog additionally includes 449 missing training entities recovered via Wikidata tr/mul/en label fallback; its frozen snapshot is in wikidata-supplement.jsonl in the full bundle. No test labels were used for this repair. Selected train/validation/test documents: 28,310 / 2,350 / 2,353. Seed: 42.

Full training settings, actual effective counts and validation histories are in training.json for each component. The bundle contains source, manifests, evaluation reports and a reproducible full-corpus recipe. The full recipe was not run for this pilot. GPU floating-point kernels and parallel HNSW construction may differ slightly across machines.

Limitations and license

Turkish case and apostrophe features are shared between training and inference; this is not a full morphological disambiguator. Mention supervision is weak and not exhaustive. The corpus, automatic annotations, translated descriptions and historical catalog can contain errors. Performance can degrade on new domains, new entities, common nouns and ambiguous mentions. Confidence is calibrated for a Q-ID given a supplied span, not for mention correctness, and may shift by domain. Abstention is threshold-based; this is not a trained NIL detector.

Corpus-derived artifacts and these checkpoints are distributed under CC BY-SA 4.0, with TurkLink attribution. Source code is Apache-2.0. Pretrained model licenses/notices remain applicable: MiniLM (Apache-2.0), BERTurk (MIT). See NOTICE and included licenses. No user text was sent to an external LLM during training.