CyberSense — Thai SMS Scam Classifier
WangchanBERTa fine-tuned to classify Thai SMS messages as scam (1) or legitimate (0). Part of CyberSense.
Input format
The model was trained with the sender name prefixed to the message:
Sender: <sender name> | <message text>
Use Sender: SMS | <message text> when the sender is unknown.
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
repo = "FRAMERIA/cybersense-wangchanberta"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
text = "Sender: SMS | ท่านมีพัสดุค้างที่ไปรษณีย์ กรุณายืนยันที่อยู่ คลิก cutt.ly/9xbb"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
scam_prob = torch.softmax(model(**inputs).logits, dim=1)[0][1].item()
print(f"Scam probability: {scam_prob:.0%}")
Training
- Dataset: 700 Thai SMS messages (350 scam / 350 legitimate), split 70/15/15 (train/validation/test, stratified)
- 5 epochs, batch size 16, best checkpoint selected by F2-score on the scam class (recall weighted 2× over precision)
Evaluation (held-out test set, 105 messages)
| Model | Accuracy | Scam recall |
|---|---|---|
| TF-IDF + Logistic Regression (text) | 98.1% | 98.1% |
| This model | 100% | 100% |
Limitations
The dataset is small and relatively clean — a simple TF-IDF baseline already reaches 98%. These scores do not guarantee the same accuracy on real-world messages. Use as a demo and research artifact, not as a sole safeguard.
- Downloads last month
- 17