distilbert-multilingual-vifn

This model is distilbert-base-multilingual-cased fine-tuned for binary Vietnamese fake-news classification on the text-only ViFN benchmark.

Evaluation protocol

• Dataset size: 1,406 examples.
• Fixed splits: 1,124 train / 141 development / 141 test.
• Split seed: 42, stratified by label with exact duplicate groups kept in one split.
• Fine-tuning seeds: [42, 22, 202].
• Training: 3 epoch(s), AdamW, learning rate 2e-05, weight decay 0.01, warmup ratio 0.1.
• Effective train batch size: 8.
• Maximum sequence length: 256.
• Raw Vietnamese text was tokenized directly with the released tokenizer; no external word segmentation.
• No class weighting, resampling, external metadata, images, engagement features, or test-time model selection.
• Checkpoints are selected by development Macro-F1. The representative published checkpoint is seed 202, selected only by development Macro-F1.

Results

Test metrics are reported as mean ± sample standard deviation over seeds [42, 22, 202].

Metric	Mean ± std
Test Macro-F1	0.8226 ± 0.0072
Test accuracy	0.8227 ± 0.0071
Test macro precision	0.8237 ± 0.0061
Test macro recall	0.8229 ± 0.0070
Development Macro-F1	0.7986 ± 0.0043

Per-seed results

seed	dev_macro_f1	test_macro_f1	test_accuracy	micro_batch_size	gradient_accumulation_steps
22.000000	0.793662	0.815371	0.815603	8.000000	1.000000
42.000000	0.800928	0.822659	0.822695	8.000000	1.000000
202.000000	0.801329	0.829779	0.829787	8.000000	1.000000

Label mapping

{
  "0": "0",
  "1": "1"
}

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "BaoNhan/distilbert-multilingual-vifn"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Đây là nội dung tin tức tiếng Việt cần phân loại."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    probabilities = model(**inputs).logits.softmax(dim=-1)[0]
predicted_id = int(probabilities.argmax())
print(model.config.id2label[predicted_id], probabilities.tolist())

Files

• aggregate_metrics.json: complete aggregate metrics and training manifest.
• artifacts/per_seed_results.csv: one row per fine-tuning seed.
• artifacts/seed_*_confusion_matrix.csv: confusion matrix for each seed.
• artifacts/seed_*_classification_report.json: per-class metrics.
• artifacts/seed_*_test_predictions.csv: IDs, gold/predicted labels and probabilities; raw text is excluded.

Limitations

ViFN is small and domain-specific. Performance may not transfer to newly emerging misinformation, other Vietnamese writing styles, or texts requiring image/source/engagement evidence. The model predicts from linguistic content only and should not be treated as a factual verification system.

Dataset citation

@article{huynh2025vifn,
  title={Utilizing Transformer Models To Detect Vietnamese Fake News on Social Media Platforms},
  author={Huynh, Anh-Tuan and Tran, Phuoc},
  journal={KSII Transactions on Internet and Information Systems},
  volume={19},
  number={2},
  pages={472--487},
  year={2025},
  doi={10.3837/TIIS.2025.02.006}
}