yazidsupriadi/bot-detection-mbert

IndoBERT-GRU for Bot Detection on Platform X

This model uses MBERT to classify Twitter/X accounts as bots or humans based on their textual content and numeric features.

Model Architecture

• Base Model: bert-base-multilingual-cased
• Classifier: 2-layer feedforward with ReLU + Sigmoid

Dataset

The dataset includes Indonesian tweets with labels (0 for human, 1 for bot), along with additional numeric features such as favorite_count, retweet_count, reply_count, and quote_count.

Training Details

• Loss Function: Binary Cross Entropy (BCELoss)
• Optimizer: AdamW (lr = 2e-5)
• Epochs: 10
• Max Token Length: 128
• Batch Size: 16

Training Log

Epoch	Train Loss	Val Accuracy	Precision	Recall	F1-score
1	0.2911	0.8733	0.8611	0.8916	0.8761
2	0.1596	0.9416	0.9445	0.9388	0.9417
3	0.0863	0.9704	0.9709	0.9703	0.9706
4	0.0587	0.9796	0.9801	0.9792	0.9797
5	0.0377	0.9884	0.9894	0.9876	0.9885
6	0.0324	0.9904	0.9888	0.9922	0.9905
7	0.0219	0.9937	0.9944	0.9930	0.9937
8	0.0223	0.9936	0.9938	0.9934	0.9936
9	0.0185	0.9936	0.9937	0.9937	0.9937
10	0.0150	0.9957	0.9958	0.9958	0.9958

Evaluation Results

Final Validation Accuracy: 0.9370

Final Precision: 0.9333 Final Recall: 0.9408 Final F1-score: 0.9370

Confusion Matrix

Training Phase

Classification Report

              precision    recall  f1-score   support

         0.0       0.94      0.93      0.94      2008
         1.0       0.93      0.94      0.94      1992

    accuracy                           0.94      4000
   macro avg       0.94      0.94      0.94      4000
weighted avg       0.94      0.94      0.94      4000