Tanor/sr_judig2026_teslaxlm

Tesla JuDig2026 — Serbian linguistic annotation and entity linking

A complete Serbian spaCy pipeline built on TE-SLA/TeslaXLM. It predicts sentence boundaries, Universal POS tags, lemmas, nine named-entity classes and Wikidata identifiers. The release bundles the trained pipeline, tokenizer, lookup KB and its required reader/scorer registrations.

Version: 1.0.0 · Publisher: Tanor · Training: October 22, 2026.

Test results at a glance

Task	Metric	Result (%)
Sentence detection	F1	87.45
UPOS tagging	Token accuracy	96.36
POS annotation	Token accuracy	96.36
Lemmatization	Token accuracy	95.94
Named entity recognition	Micro F1	83.90
Entity linking	Known-QID gold-span F1	97.04

Evaluation scope matters: these results use original, pretokenized test documents. Predictions receive no gold POS tags, lemmas or sentence boundaries. The NEL metric uses gold entity spans and known gold QIDs, with a KB built from all train/dev/test annotations. It is neither end-to-end linking F1 nor evidence of generalization to entities or aliases absent from that KB. Unknown links are excluded; the corpus supplies no explicit NIL gold for measuring NIL rejection. Raw-text tokenization and external-text performance have not been benchmarked for this release.

Installation and use

Install the wheel from this repository in a compatible Python environment:

pip install "https://huggingface.co/Tanor/sr_judig2026_teslaxlm/resolve/main/sr_judig2026_teslaxlm-1.0.0-py3-none-any.whl"

The tested environment is Python 3.11, spaCy 3.8.11, spacy-transformers 1.3.9, Transformers 4.49.0 and PyTorch 2.7.1. Install an appropriate PyTorch build for your CPU/CUDA environment; the packaged model supports CPU inference.

import sr_judig2026_teslaxlm

nlp = sr_judig2026_teslaxlm.load()
doc = nlp("Profesor Marko radi u Beogradu. Ovo je nova rečenica.")

for sent in doc.sents:
    print(sent.text)
for token in doc:
    print(token.text, token.pos_, token.lemma_)
for ent in doc.ents:
    print(ent.text, ent.label_, ent.kb_id_)

ent.kb_id_ is a Wikidata QID when a candidate is selected, or NIL when none is available. A selected QID is a prediction, not a verified identity. Install and load the Python package as shown: its initialization registers the custom scorer and corpus reader automatically, without importing this training repository.

Pipeline

transformer → senter → tagger → attribute_ruler → trainable_lemmatizer → ner → entity_linker

The tagger learns UPOS, and the attribute ruler copies predicted TAG into token.pos_. TAG and POS accuracy therefore measure the same prediction. The lemmatizer learns edit trees. Legacy rich XPOS and morphological features are retained in corpus provenance, but this model does not predict them.

The 17 learned UPOS labels are ADJ ADP ADV AUX CCONJ DET INTJ NOUN NUM PART PRON PROPN PUNCT SCONJ SYM VERB X. The NER classes are listed below.

Corpus and split design

Training uses the reviewed JuDig2026 v2 snapshot with approved lemma/UPOS corrections, including the reviewed Švejk replacement. Six problematic source sentences (122 tokens) were quarantined before packaging. The raw corpus is not distributed in this model repository.

Split	Documents	Sentences	Tokens	Entities	QID-linked	Unspecified link
train	3,605	35,712	851,278	47,012	30,178	16,834
dev	448	4,400	75,414	6,129	5,115	1,014
test	660	6,553	108,219	2,615	1,927	688
all	4,713	46,665	1,034,911	55,756	37,220	18,536

Source families are assigned together: source files, source families, original sentence IDs and document IDs do not cross splits. This is a source-family-stratified split, not a random sentence split; its actual ratios differ from the initial 80/10/10 targets. The revision ensures all 17 POS labels occur in train and all nine NER classes occur in each split. All 72 SYM examples belong to one indivisible family and occur only in train. The source's 210 repeated sentence-ID occurrences are retained within a single split, not silently removed.

Class	Meaning	Train	Dev	Test
PERS	Persons	3601	1967	1062
ROLE	Roles/professions	13757	1915	507
LOC	Locations	8929	1342	262
ORG	Organizations	10445	232	528
DEMO	Demonyms/groups	4001	476	84
WORK	Named works	5739	42	114
EVENT	Events	322	23	54
PRODUCT	Products	193	126	3
IDEO	Ideologies	25	6	1

Missing gold is preserved rather than guessed: train has 145 masked lemma tokens and 15 masked POS/TAG tokens; dev has three masked POS/TAG tokens. NER masks are also preserved. Corpus binary hashes, counts and split audit are provided in evaluation/; document-level source text is excluded.

Training recipe

Setting	Value
Encoder initialization	Original pretrained TeslaXLM; fresh task heads
Random seed	42
Transformer span window / stride	128 / 96 spaCy tokens
Optimizer	Adam, weight decay 0.01, gradient clip 1.0
Learning rate	2e-5, 250-step warmup, linear schedule to 20,000
Gradient accumulation	3
Dropout	0.1
Dev evaluation frequency	Every 200 steps
Early-stopping patience	1,600 steps
Maximum / final logged step	20,000 / 7,000
Best checkpoint selection	Equal weights: sentence F1, TAG accuracy, lemma accuracy, NER F1, NEL F1
Hardware	One NVIDIA RTX 4090, 24 GB VRAM
Main training duration	About 2 hours 45 minutes, including initialization/checkpoint saving

The initial step-zero full-dev evaluation is skipped; subsequent scheduled evaluations are real. The released artifact is model-best, selected on development performance, rather than model-last. Full dev alignment passed on 448 documents / 75,414 tokens, with zero unaligned alphanumeric tokens. The 128/96 windows accommodate expansion from spaCy tokens to encoder wordpieces.

Knowledge base and description preparation

The bundled KB contains 4,013 QIDs, 16,803 alias/entity rows, and 1,024-dimensional TeslaXLM vectors. Corpus mention forms, including declined forms present in the corpus, provide aliases; Latin/Cyrillic variants are added. The pipeline does not generate every possible grammatical case form.

Per the experiment protocol, the KB uses the all corpus file, combining train, dev and test. Only the training split contributes gradient updates. All known gold QID mentions in every split have the correct QID among their exact-alias candidates. This is intentionally a known-KB evaluation.

Description preparation reused 3,495 records from an earlier enriched store and fetched missing records from Wikidata/Wikipedia. The encoder uses 3,922 descriptions and 91 canonical-name fallbacks, with no bare-QID fallback vectors. Descriptions are preferred to names. Short translated definitions remain valid, so final encoding accepts nonempty description text.

When external translation was rate limited, Codex translated 150 unique English texts into Serbian Latin, used by 156 QID records. Original/target texts and source hashes are retained in the preparation workspace; the public bundle includes their summary, not full Wikipedia text. Legacy per-record fetch language/time and article attribution are not completely available; the source store and its hash are recorded. Cached legacy descriptions are reused as stored. Translation quality was not independently benchmarked.

Linking uses context and alias priors, with threshold = null: choose a KB candidate when available, otherwise predict NIL. The vector loss ignores unspecified links. Evaluation also excludes those links rather than treating them as correct/incorrect NIL decisions.

Detailed test evaluation

Named entity recognition (%)

Class	Gold support	Precision	Recall	F1
PERS	1062	96.31	93.31	94.79
ROLE	507	72.95	64.89	68.68
LOC	262	87.19	93.51	90.24
ORG	528	85.71	71.59	78.02
DEMO	84	80.90	85.71	83.24
WORK	114	71.05	71.05	71.05
EVENT	54	100.00	31.48	47.89
PRODUCT	3	0.00	0.00	0.00
IDEO	1	0.00	0.00	0.00

Entity linking: gold spans, known QIDs (%)

Class	Gold support	Precision	Recall	F1
PERS	See note	99.67	99.67	99.67
ROLE	See note	93.16	93.16	93.16
LOC	See note	93.12	93.12	93.12
ORG	See note	97.78	97.78	97.78
DEMO	See note	87.50	87.50	87.50
WORK	See note	97.83	97.83	97.83
EVENT	See note	100.00	100.00	100.00
PRODUCT	See note	—	—	—
IDEO	See note	—	—	—

NEL support is the known-QID subset, not the total NER support shown above; there are 1,927 known-QID test mentions overall. A dash means that no scored known-QID example contributes that class, not a measured zero. Raw scorer outputs and KB coverage are included in evaluation/.

Intended use and limitations

Use for Serbian text annotation, corpus exploration, indexing and research with human review. Latin and Cyrillic inputs are accepted; the reported test does not establish equal performance across scripts, genres or unseen domains.

PERS and LOC are stronger than ROLE and EVENT in this test. IDEO and PRODUCT have only one and three test entities respectively, and both score zero; this is limited evidence, not robust class-level evaluation. Sentence detection is imperfect and can affect raw-text downstream behavior. No external srpGeography evaluation has been run on this JuDig release.

Only one seed is reported. Earlier JuDig scores used different splits and a train-only KB, so their NEL difference cannot be attributed to descriptions alone. Known aliases may still resolve to the wrong entity. Entities absent from the bundled KB cannot be linked to new QIDs without KB expansion. The null linker threshold is not calibrated for NIL rejection. POS/lemma scores assume source tokenization and do not measure raw-text tokenization quality.

Reproducibility and contents

The wheel bundles the complete model, KB and portable custom registrations. training_config.cfg, training manifest, counts/masks, corpus hashes, alignment check, training log and test JSON document the experiment. Inspection copies of the training launchers are in training/; rerunning requires the original corpus, preparation utilities and runtime. This is not a standalone corpus release. SHA256SUMS, release_manifest.json and verification.json identify and validate the release artifacts.

License and attribution

CC BY-SA 4.0. See LICENSE and LICENSES_SOURCES. Credit Tanor for this fine-tuned pipeline and TE-SLA for TeslaXLM. Wikidata provides QIDs and structured descriptions; Wikipedia provides summary text used in KB preparation. No additional license for the underlying private corpus is asserted by this model card.