distilbert-multilingual-UIT-ViON

This model is distilbert-base-multilingual-cased fine-tuned for UIT-ViON online-news topic classification on UIT-ViON.

Evaluation protocol

• Dataset size: 260,000 examples.
• Original published fixed splits: 208,000 train / 26,000 development / 26,000 test. No rows are moved between splits.
• Dataset source: https://github.com/kh4nh12/UIT-ViON-Dataset.
• The split audit records exact and near duplicates; 797 normalized exact-text group(s) span published splits and are preserved for comparability.
• Every label contributes exactly 16,000/2,000/2,000 examples to train/development/test. Label ID 7 is displayed as HEALTH, correcting the HEATH typo in the repository README without changing the class ID.
• Fine-tuning seeds: [42, 22, 202].
• Training: 3 epoch(s), AdamW, learning rate 2e-05, weight decay 0.01, warmup ratio 0.1.
• Effective train batch size: 8.
• Maximum sequence length: 256.
• Input mode: official titles with word-segmentation underscores converted to spaces.
• No class weighting, resampling, external metadata, images, engagement features, or test-time model selection.
• Checkpoints are selected by development Macro-F1. The representative published checkpoint is seed 202, selected only by development Macro-F1.

Results

Test metrics are reported as mean ± sample standard deviation over seeds [42, 22, 202].

Metric	Mean ± std
Test Macro-F1	0.7571 ± 0.0012
Test accuracy	0.7586 ± 0.0009
Test macro precision	0.7572 ± 0.0013
Test macro recall	0.7586 ± 0.0009
Development Macro-F1	0.7520 ± 0.0014

Per-seed results

seed	dev_macro_f1	test_macro_f1	test_accuracy	micro_batch_size	gradient_accumulation_steps
22.000000	0.750645	0.756316	0.758000	8.000000	1.000000
42.000000	0.752043	0.758475	0.759615	8.000000	1.000000
202.000000	0.753357	0.756655	0.758038	8.000000	1.000000

Label mapping

{
  "0": "TECHNOLOGY",
  "1": "TRAVEL",
  "2": "EDUCATION",
  "3": "ENTERTAINMENT",
  "4": "SCIENCE",
  "5": "BUSINESS",
  "6": "LAW",
  "7": "HEALTH",
  "8": "WORLD",
  "9": "SPORT",
  "10": "NEWS",
  "11": "VEHICLE",
  "12": "LIFE"
}

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "BaoNhan/distilbert-multilingual-UIT-ViON"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Đây là nội dung tiếng Việt cần phân loại."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)
with torch.no_grad():
    probabilities = model(**inputs).logits.softmax(dim=-1)[0]
predicted_id = int(probabilities.argmax())
print(model.config.id2label[predicted_id], probabilities.tolist())

Files

• aggregate_metrics.json: aggregate metrics and training manifest.
• artifacts/per_seed_results.csv: one row per fine-tuning seed.
• artifacts/seed_*_confusion_matrix.csv: confusion matrix for each seed.
• artifacts/seed_*_classification_report.json: per-class metrics.
• artifacts/seed_*_test_predictions.csv: IDs, gold/predicted labels and probabilities; raw text is excluded.

Limitations

UIT-ViON contains semi-automatically annotated, word-segmented news titles collected from six Vietnamese online newspapers in a particular period. Topic balance is artificial and does not represent real-world topic prevalence. Performance may not transfer to full articles, new outlets, newer events, or changed editorial taxonomies. Source URLs are retained only for audit and are never model inputs.

Dataset citation

@inproceedings{tran2021uitvion,
  author={Tran, Khanh Quoc and Trinh, Phap Ngoc and Tran, Khoa Nguyen-Anh and Le, An Tran-Hoai and Ha, Luan Van and Nguyen, Kiet Van},
  title={An Empirical Investigation of Online News Classification on an Open-domain, Large-scale and High-quality Dataset in Vietnamese},
  booktitle={New Trends in Intelligent Software Methodologies, Tools and Techniques},
  year={2021},
  pages={367--379},
  publisher={IOS Press},
  doi={10.3233/FAIA210036}
}