cafebert-ViHSD

This model is uitnlp/CafeBERT fine-tuned for ViHSD hate speech detection on ViHSD.

Evaluation protocol

• Dataset size: 33,400 examples.
• Original published fixed splits: 24,048 train / 2,672 development / 6,680 test.
• No rows were moved between the published splits.
• Fine-tuning seeds: [22, 42, 202].
• Training: 3 epoch(s), AdamW.
• Learning rate: 2e-05.
• Weight decay: 0.01.
• Warmup ratio: 0.1.
• Training batch size: 8.
• Maximum sequence length: 256.
• Input mode: raw Vietnamese social-media text.
• The published checkpoint is seed 22.

Results

Metrics are reported as mean ± sample standard deviation over the available completed seeds.

Metric	Mean ± std
Test Macro-F1	0.6430 ± 0.0197
Test accuracy	0.8739 ± 0.0064
Test macro precision	0.6718 ± 0.0203
Test macro recall	0.6312 ± 0.0160
Development Macro-F1	0.6530 ± 0.0118

Per-seed results

seed	dev_macro_f1	test_macro_f1	test_accuracy
22.000000	0.666226	0.656446	0.877246
42.000000	0.649098	0.620422	0.866467
202.000000	0.643704	0.652192	0.877994

Label mapping

{
  "0": "CLEAN",
  "1": "OFFENSIVE",
  "2": "HATE"
}

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "BaoNhan/cafebert-ViHSD"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Đây là nội dung tiếng Việt cần phân loại."

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=256,
)

with torch.no_grad():
    probabilities = model(**inputs).logits.softmax(dim=-1)[0]

predicted_id = int(probabilities.argmax())
print(model.config.id2label[predicted_id], probabilities.tolist())

Limitations

ViHSD is class-imbalanced and reflects Vietnamese social-media language from a particular collection period. Performance may not transfer directly to new platforms, dialects, code-switching patterns, irony, or emerging slang. Predictions should not be the sole basis for moderation or punitive decisions.

Dataset citation

@InProceedings{10.1007/978-3-030-79457-6_35,
  author={Luu, Son T. and Nguyen, Kiet Van and Nguyen, Ngan Luu-Thuy},
  title={A Large-Scale Dataset for Hate Speech Detection on Vietnamese Social Media Texts},
  booktitle={Advances and Trends in Artificial Intelligence. Artificial Intelligence Practices},
  year={2021},
  publisher={Springer International Publishing},
  pages={415--426},
  doi={10.1007/978-3-030-79457-6_35}
}