IndoBERT-Sentiment-Analysis

This model is a fine-tuned version of indobenchmark/indobert-base-p1 on the indonlu dataset. It achieves the following results on the evaluation set:

• Loss: 0.4366
• Accuracy: 0.9413
• F1 Score: 0.9411

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Score
0.348	0.2726	500	0.2575	0.9238	0.9234
0.316	0.5453	1000	0.3581	0.9167	0.9152
0.2988	0.8179	1500	0.3070	0.9198	0.9197
0.2517	1.0905	2000	0.3738	0.9206	0.9199
0.1715	1.3631	2500	0.3353	0.9325	0.9320
0.1589	1.6358	3000	0.3510	0.9286	0.9282
0.1752	1.9084	3500	0.2935	0.9341	0.9341
0.0991	2.1810	4000	0.3740	0.9373	0.9376
0.0677	2.4537	4500	0.3930	0.9294	0.9289
0.0711	2.7263	5000	0.3676	0.9333	0.9332
0.1037	2.9989	5500	0.3072	0.9373	0.9371
0.0323	3.2715	6000	0.3811	0.9325	0.9325
0.0413	3.5442	6500	0.4210	0.9373	0.9367
0.029	3.8168	7000	0.4036	0.9373	0.9371
0.0235	4.0894	7500	0.4489	0.9365	0.9364
0.01	4.3621	8000	0.4440	0.9397	0.9395
0.0079	4.6347	8500	0.4477	0.9381	0.9380
0.012	4.9073	9000	0.4340	0.9413	0.9411

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0
• Datasets 4.2.0
• Tokenizers 0.22.1