ivlcic/snerta-8l-base

Snerta-8l Base

Snerta-8l Base is a multilingual named entity recognition model for eight Slavic languages. It is a 278M-parameter microsoft/mdeberta-v3-base model fine-tuned on the complete, deduplicated training pool of the eight manually annotated language datasets prepared for the paper When Is Multilingual Transfer Beneficial for Slavic Named Entity Recognition?

Unlike Snerta-12l Base, this checkpoint does not use the lower-confidence WikiANN / PAN-X auxiliary datasets for Bosnian, Macedonian, Slovak, or Albanian.

The model recognises four coarse entity types in IOB2 format:

Type	Meaning
PER	Person
LOC	Location
ORG	Organization
MISC	Miscellaneous entity, including source-specific labels mapped into the shared category

The underlying nine labels are O, B-LOC, I-LOC, B-MISC, I-MISC, B-ORG, I-ORG, B-PER, and I-PER.

Intended use

The model is intended for general-purpose named entity extraction from Bulgarian, Czech, Croatian, Polish, Russian, Slovene, Serbian, and Ukrainian text. It is particularly useful when one shared model is preferable to maintaining a separate model for every language and training exclusively on the eight manually annotated language pools.

It is suitable as a component in search, document enrichment, media monitoring, corpus analysis, and entity-centric analytics. It should not be used as the sole mechanism for anonymisation, access control, legal decisions, or other high-stakes applications. Validate it on the target domain and language before deployment.

Usage

Install Transformers and create a token-classification pipeline with simple entity aggregation:

from transformers import pipeline

model_id = "ivlcic/snerta-8l-base"
ner = pipeline(
    task="token-classification",
    model=model_id,
    tokenizer=model_id,
    aggregation_strategy="simple",
)

text = (
    "Janez Novak... Metka Kralj,,. in Boris A. Novak živijo v Ljubljani "
    "in delajo za Microsoft."
)

for entity in ner(text):
    span = text[entity["start"]:entity["end"]]
    print(entity["entity_group"], span, round(float(entity["score"]), 3))

The example identifies Janez Novak, Metka Kralj, and Boris A. Novak as PER, Ljubljani as LOC, and Microsoft as ORG.

For applications that need to preserve an existing tokenisation—including whitespace and punctuation—the model can also be called with pre-split tokens, as in the CaNNopy inference example:

import re

tokens = re.findall(r"\s+|\w+|[^\w\s]", text, flags=re.UNICODE)
entities = ner(
    tokens,
    is_split_into_words=True,
    delimiter="",
)[0]

for entity in entities:
    span = text[entity["start"]:entity["end"]]
    print(entity["entity_group"], span)

Inputs longer than 256 model tokens should be split into overlapping windows by the application. Take care to reconcile entities that cross window boundaries.

Training data

The training corpus and its derived splits are not distributed with this model or in the supplementary repository. Obtain each source corpus from its original provider and follow its license or access terms. For this model, source corpora belonging to the same language key were pooled before an 80/10/10 sentence-level train/validation/test split.

Language	Code	Source corpus or corpora	Prepared sentences before splitting and deduplication	Role in the paper
Bulgarian	bg	BSNLP / SlavNER	18,333	Main
Czech	cs	BSNLP / SlavNER; CNEC 2.0	20,864	Main
Croatian	hr	hr500k 1.0	24,794	Main
Polish	pl	BSNLP / SlavNER	20,423	Main
Russian	ru	BSNLP / SlavNER	25,141	Main
Slovene	sl	BSNLP / SlavNER; ssj500k 2.3, ELEXIS-WSD 1.0, and SentiCoref 1.0 from SUK 1.0	48,701	Main
Serbian	sr	SETimes.SR 1.0	3,891	Main
Ukrainian	uk	BSNLP / SlavNER; NER-UK 2.0	24,424	Main

The checkpoint uses the full, unbalanced training pool for these eight language keys. It was not trained on the auxiliary WikiANN / PAN-X pools for Bosnian, Macedonian, Slovak, or Albanian. The separate Croatian WikiANN data key used for the paper's source-quality ablation was also not used.

After deduplication, the model's eight-language pool contains 137,009 training sentences, 18,030 validation sentences, and 18,414 test sentences. The training split contains approximately 2.78 million word-level tokens. The training pipeline labels only the first model subtoken of each original word and ignores later subtokens when computing the loss and metrics.

Deduplication and relationship to prior work

Unlike the precursor ivlcic/sour-sarma, this model was trained on sentence-deduplicated data. Duplicate matching was performed independently within each language data key on the NFKC-normalized, case-folded token sequence. When a duplicate crossed splits, the test occurrence was retained over validation and training, and validation was retained over training. Repetitions within a split were also removed.

Across the eight language keys used for this checkpoint, 13,118 of 186,571 prepared sentence instances were removed. Deduplication prevents exact sentence leakage but does not remove paraphrases, near-duplicates, or related sentences from the same document.

There is also an intentional label-space difference from the paper experiments. The controlled experiments in the paper use only PER, LOC, and ORG, mapping unsupported labels—including MISC—to O. This release retains MISC because it is intended for a broader operational usage scenario. Consequently, the benchmark below should not be compared directly with the paper's three-type, eight-evaluation-language, three-seed macro averages.

Source data terms

The combined training corpus does not have a single uniform data license. Each source retains its own license, access conditions, and citation requirements:

• SUK 1.0, hr500k 1.0, and SETimes.SR 1.0 are distributed under CC BY-SA 4.0.
• CNEC 2.0 is distributed under CC BY-NC-SA 3.0.
• NER-UK 2.0 is distributed under CC BY-NC-SA 4.0.
• The BSNLP / SlavNER release used for training does not provide an unambiguous data-redistribution license.

This section documents training-data provenance; it does not grant access to or permission to redistribute any source corpus. The Apache-2.0 license in this model repository applies to the model release and does not replace source-dataset terms.

Training procedure

The model was initialised from microsoft/mdeberta-v3-base and trained in FP32 with the following configuration:

Hyperparameter	Value
Maximum sequence length	256
Epochs	20
Per-device training batch size	16
Gradient accumulation steps	2
Effective training batch size	32
Per-device evaluation batch size	32
Optimizer	PyTorch AdamW
Learning rate	2e-5
Learning-rate schedule	Linear
Warmup ratio	0.06
Weight decay	0.01
Maximum gradient norm	1.0
Dropout	0.10
Random seed	2611
Checkpoint selection	Best validation entity-level F1

The run evaluated and saved a checkpoint after every epoch. The selected checkpoint was from epoch 15, with validation F1 92.39. Training completed 85,620 optimiser steps; the recorded training runtime was approximately 4 hours 21 minutes. The run used PyTorch 2.11.0, Transformers 5.8.0, and seqeval 1.2.2.

The learning rate and dropout follow the mDeBERTa-v3 setting selected by the paper's bounded validation sweep. In the Transformers mDeBERTa implementation, the configured dropout maps to hidden dropout and therefore affects the encoder layers as well as the token-classification head.

Evaluation

The released checkpoint was evaluated once on the combined deduplicated test split for the same eight language keys used in training. Metrics were computed with seqeval at entity level: a prediction is correct only when both its span and entity type match exactly. Precision, recall, and F1 below are micro-averaged over entities; accuracy is calculated over labeled word positions.

Entity type	Precision	Recall	F1
Overall	91.80	92.82	92.31
LOC	95.07	96.00	95.53
MISC	84.10	83.79	83.95
ORG	89.84	92.32	91.06
PER	93.62	93.95	93.79

Token accuracy is 98.78% and test loss is 0.0917. These values come from the saved CaNNopy evaluation artefact for this checkpoint.

Limitations and biases

• The benchmark uses a random sentence-level split of pooled source corpora. It does not measure cross-document, temporal, topical, or cross-domain generalisation.
• Results come from one training seed and are aggregated across all eight languages. Performance can vary substantially by language and domain, and the aggregate score can obscure lower-performing subsets.
• The corpora differ in domain, segmentation, annotation density, provenance, and label definitions. MISC is especially heterogeneous across sources.
• The full-pool training recipe is intentionally unbalanced, so larger language corpora contribute more training examples.
• Exact sentence deduplication does not remove near-duplicates or guarantee document-level separation.
• Names, organisations, locations, and demographic groups that are rare or absent in the training corpora may be recognised less reliably. False positives and false negatives should be expected.
• Although the base encoder is multilingual, this fine-tuned model was trained and evaluated only on the eight listed languages. Performance on other languages is unknown.
• Sequences are truncated at 256 model tokens unless the calling application implements windowing.

Reproducibility

• Training and evaluation code: ivlcic/cannopy
• Base checkpoint: microsoft/mdeberta-v3-base
• Twelve-language companion model: ivlcic/snerta-12l-base
• Precursor model: ivlcic/sour-sarma