Edit model card

fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-without-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.5397
  • Exact Match: 47.8725
  • F1: 64.1189

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Exact Match F1
1.8628 0.5 463 1.7727 40.7249 56.9043
1.6706 1.0 926 1.6163 44.3912 61.0635
1.5058 1.5 1389 1.5655 45.4339 61.5089
1.4661 2.0 1852 1.5130 46.9055 63.5850
1.3171 2.5 2315 1.5077 47.1914 63.4762
1.3258 3.0 2778 1.4981 47.6034 64.3797
1.1835 3.5 3241 1.5171 47.7043 64.1444
1.1946 4.0 3704 1.5333 47.6539 64.3327
1.0904 4.5 4167 1.5397 47.8725 64.1189

Framework versions

  • Transformers 4.26.1
  • Pytorch 1.13.1+cu117
  • Datasets 2.2.0
  • Tokenizers 0.13.2
Downloads last month
8