kmdb_ner_v5

Hungarian named entity recognition for the K-Monitor press database. It does two jobs at once: it finds people, organisations and places in an article, and it marks which of them are the ones an editor would actually record for that article.

Labels are B-/I- + {POS,NEG}_{PER,ORG,LOC}. POS means the entity is relevant to the article's case and belongs in the database; NEG means it is merely mentioned.

Finetuned from SZTAKI-HLT/hubert-base-cc on 58,524 articles from K-Monitor/kmdb_base: spaCy (hu_core_news_lg) supplies the entity spans, and an entity is labelled POS when it matches one of the names an editor tagged on that article.

Results

Entity-level seqeval scores on 4,161 held-out articles. The honest comparison against the previous boapps/kmdb_ner_model is the second column: that checkpoint was trained with an unseeded split, so most of the general test set was in its training data, whereas these articles were added to the database after it was published.

model	test F1	F1 on articles the old model never saw
boapps/kmdb_ner_model	0.8244*	0.6684
kmdb_ner_v5	0.8171	0.7299

* inflated by train/test leakage in the earlier checkpoint.

Per class, on those unseen articles:

label	precision	recall	F1	previous F1
NEG_LOC	0.7817	0.7075	0.7427	0.7208
NEG_ORG	0.7530	0.7587	0.7558	0.7176
NEG_PER	0.7298	0.6752	0.7014	0.6505
POS_LOC	0.6219	0.7128	0.6642	0.5729
POS_ORG	0.6469	0.7130	0.6784	0.5270
POS_PER	0.6971	0.8029	0.7463	0.6484
micro avg	0.7242	0.7356	0.7299	0.6684

The largest gain is on POS_ORG (0.5270 → 0.6784): the previous training data matched gold names by exact substring, so tagged organisations whose database name carries a suffix — Hunguest Hotels Zrt., Honvédelmi Minisztérium (HM) — were labelled NEG and the model learned to ignore them.

Usage

from transformers import AutoTokenizer, pipeline

classifier = pipeline(
    "ner",
    model="K-Monitor/kmdb_ner_v5",
    aggregation_strategy="first",
    stride=8,
    tokenizer=AutoTokenizer.from_pretrained(
        "SZTAKI-HLT/hubert-base-cc", model_max_length=512
    ),
)
classifier(article_text)

Adjacent fragments of one entity have to be merged afterwards (huBERT splits detached Hungarian suffixes, e.g. Budapest + -en); when merging, keep the entity type and treat the result as POS if either fragment is POS. The reference implementation is join_entities in sajtoadatbazis-automat (webapp/auto_kmdb/processors/NERProcessor.py).

Limitations

The POS/NEG supervision is derived automatically by matching spaCy entities against editor-tagged names, so it inherits spaCy's boundary errors and the matching heuristic's mistakes; it is not hand-annotated. Scores on articles from after the training window (0.73 F1) are well below in-distribution scores (0.82), so expect the real-world quality to track the former.