Edit model card

sentiment-lora-r16

This model is a fine-tuned version of indolem/indobert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2631
  • Accuracy: 0.8872
  • Precision: 0.8658
  • Recall: 0.8602
  • F1: 0.8629

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 30
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20.0

Training results

Training Loss Epoch Step Validation Loss Accuracy Precision Recall F1
0.5544 1.0 122 0.5252 0.7218 0.6705 0.6807 0.6746
0.4974 2.0 244 0.4521 0.7845 0.7792 0.6650 0.6840
0.4206 3.0 366 0.3642 0.8296 0.8034 0.7694 0.7829
0.3657 4.0 488 0.3376 0.8471 0.8184 0.8068 0.8122
0.3188 5.0 610 0.3198 0.8496 0.8202 0.8136 0.8167
0.3069 6.0 732 0.3127 0.8546 0.8239 0.8272 0.8255
0.2838 7.0 854 0.3053 0.8672 0.8449 0.8285 0.8360
0.2699 8.0 976 0.2976 0.8747 0.8647 0.8238 0.8404
0.2614 9.0 1098 0.2887 0.8647 0.8398 0.8292 0.8342
0.2515 10.0 1220 0.2852 0.8596 0.8316 0.8282 0.8298
0.2453 11.0 1342 0.2800 0.8697 0.8411 0.8478 0.8443
0.236 12.0 1464 0.2718 0.8797 0.8633 0.8399 0.8502
0.227 13.0 1586 0.2712 0.8797 0.8560 0.8524 0.8541
0.227 14.0 1708 0.2757 0.8697 0.8386 0.8603 0.8479
0.2171 15.0 1830 0.2708 0.8822 0.8530 0.8742 0.8622
0.214 16.0 1952 0.2632 0.8872 0.8658 0.8602 0.8629
0.2124 17.0 2074 0.2639 0.8822 0.8574 0.8592 0.8583
0.2166 18.0 2196 0.2632 0.8872 0.8645 0.8627 0.8636
0.2086 19.0 2318 0.2630 0.8822 0.8585 0.8567 0.8575
0.2113 20.0 2440 0.2631 0.8872 0.8658 0.8602 0.8629

Framework versions

  • Transformers 4.39.3
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.15.2
Downloads last month

-

Downloads are not tracked for this model. How to track
Unable to determine this model's library. Check the docs .

Finetuned from