Hugging Face's logo

muhammadravi251001
/
fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-base-uncased-with-ITTL-without-freeze-LR-1e-05

Question Answering
Transformers
PyTorch
TensorBoard
bert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-base-uncased-with-ITTL-without-freeze-LR-1e-05 / README.md
muhammadravi251001's picture
muhammadravi251001
update model card README.md
e43f0fb
preview code
|
raw
history blame
3.03 kB
metadata
license: mit
tags:
  - generated_from_trainer
metrics:
  - f1
model-index:
  - name: >-
      fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-base-uncased-with-ITTL-without-freeze-LR-1e-05
    results: []

fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-base-uncased-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.2740
  • Exact Match: 56.0847
  • F1: 70.6246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Exact Match F1
6.306 0.5 19 3.7982 6.5256 20.3655
6.306 1.0 38 2.8932 14.1093 26.0679
3.9254 1.5 57 2.4798 19.4004 32.1438
3.9254 2.0 76 2.2955 26.1023 37.6331
3.9254 2.5 95 2.1688 26.9841 39.2632
2.4328 3.0 114 2.0701 30.1587 41.3438
2.4328 3.5 133 1.9789 33.1570 45.0539
2.1127 4.0 152 1.8465 37.2134 48.9042
2.1127 4.5 171 1.7699 38.9771 50.9760
2.1127 5.0 190 1.6885 41.0935 54.1550
1.7875 5.5 209 1.5785 45.1499 58.6783
1.7875 6.0 228 1.4954 49.2063 62.7869
1.7875 6.5 247 1.4186 51.8519 65.7461
1.5267 7.0 266 1.3734 53.4392 67.6141
1.5267 7.5 285 1.3419 54.1446 68.2563
1.3317 8.0 304 1.3116 55.5556 69.1996
1.3317 8.5 323 1.2936 56.0847 69.9806
1.3317 9.0 342 1.2900 56.2610 70.1634
1.2556 9.5 361 1.2771 55.7319 70.1143
1.2556 10.0 380 1.2740 56.0847 70.6246

Framework versions

  • Transformers 4.26.1
  • Pytorch 1.13.1+cu117
  • Datasets 2.2.0
  • Tokenizers 0.13.2