muhammadravi251001's picture
update model card README.md
0e31d4a
metadata
license: mit
tags:
  - generated_from_trainer
metrics:
  - f1
model-index:
  - name: >-
      fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05
    results: []

fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.5867
  • Exact Match: 47.7296
  • F1: 64.3850

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 128
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Exact Match F1
1.8839 0.5 463 1.7873 39.9512 56.0205
1.6682 1.0 926 1.6243 44.2651 60.9585
1.5129 1.5 1389 1.5722 45.6609 61.7661
1.4634 2.0 1852 1.5185 47.1493 63.5348
1.3128 2.5 2315 1.5212 46.9475 63.4277
1.323 3.0 2778 1.5052 47.6118 64.2591
1.1824 3.5 3241 1.5352 47.5950 64.2896
1.2013 4.0 3704 1.5302 47.9566 64.5453
1.0842 4.5 4167 1.5678 47.5362 64.2029
1.0811 5.0 4630 1.5590 47.7632 64.1309
1.0138 5.5 5093 1.5867 47.7296 64.3850

Framework versions

  • Transformers 4.26.1
  • Pytorch 1.13.1+cu117
  • Datasets 2.2.0
  • Tokenizers 0.13.2