kmdb_classification_hubert_v5

Hungarian news article classifier for the K-Monitor press database: decides whether an article belongs in the database, i.e. whether it describes a concrete case of corruption, public procurement, public spending or transparency.

Finetuned from SZTAKI-HLT/hubert-base-cc on K-Monitor/kmdb_classification.

Input format

The model is trained on one specific string. Anything else is out of distribution:

import re

def clean(text):
    return re.sub(r"\s+", " ", text or "").strip()

def format_article(title, description, url, text=""):
    domain = ".".join(url.split("/")[2].split(".")[-2:])
    # trim_title removes trailing newspaper names such as " - Telex"
    formatted = f"{trim_title(clean(title))}\n{clean(description)}\n({domain})"
    if text:
        formatted += "\n" + clean(text)[:800]
    return formatted

Tokenize with max_length=320, truncation=True. label 1 means the article belongs in the database; the probability of that class is the score. The reference implementation lives in sajtoadatbazis-automat (webapp/auto_kmdb/processors/ClassificationProcessor.py and train/article_classification/).

Results

Held-out test split of 9,914 articles, deduplicated by URL and by title+description (40.2% positive):

model	AUPR	accuracy @0.42	precision	recall	F1
kmdb_classification_hubert_v2	0.8638	0.7119	0.590	0.926	0.7208
kmdb_classification_hubert_v5	0.9856	0.9502	0.939	0.937	0.9379

On the subset the previous version was never trained on (n=4,473), v2 scores AUPR 0.7600 / accuracy 0.4581 and v5 scores AUPR 0.9683 / accuracy 0.9215.

The score is well calibrated around the 0.42 operating point used in production; the precision/recall curve is flat between roughly 0.15 and 0.8.

threshold	accuracy	precision	recall	F1
0.15	0.9450	0.9096	0.9583	0.9333
0.374	0.9508	0.9367	0.9410	0.9388
0.42	0.9502	0.9386	0.9372	0.9379
0.60	0.9502	0.9488	0.9259	0.9372
0.80	0.9478	0.9611	0.9066	0.9330

Training

Full finetune (not LoRA): 3 epochs, lr 2e-5, linear schedule, 300 warmup steps, batch size 24, max length 320, weight decay 0.01, fp16, on a single RTX 3060. The article body is dropped from 15% of training rows so the model does not read a missing body as a label signal.

Limitations

Positive examples come from the published K-Monitor database and negatives from the autokmdb ingestion pipeline, so source and label are correlated in the training data. Whitespace normalisation and title trimming remove the mechanical traces of this (newline conventions, newspaper suffixes), and the model's output moves by 0.0004 on average when fed unnormalised text — but the correlation cannot be fully ruled out.

The companion category model K-Monitor/kmdb_classification_category_v5 is an SVM fitted on this checkpoint's CLS embedding and is not interchangeable with the SVM for earlier versions.