fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7872
• Exact Match: 46.1526
• F1: 62.7803
• Precision: 64.0856
• Recall: 68.4802

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1	Precision	Recall
1.7454	0.5	1024	1.8066	39.3967	56.5617	57.9148	64.1074
1.585	1.0	2048	1.6658	42.3134	59.7125	61.0679	67.5221
1.391	1.5	3072	1.6455	43.6763	60.8729	62.4910	67.0820
1.3937	2.0	4096	1.6125	44.9892	62.0278	63.7779	67.7429
1.1464	2.5	5120	1.6549	44.9892	61.6232	62.7582	68.2833
1.1611	3.0	6144	1.6369	45.5958	62.8723	64.0481	68.8869
1.0415	3.5	7168	1.7452	45.7038	62.3827	63.6067	68.6888
0.9972	4.0	8192	1.7086	45.7288	62.5511	64.1824	67.9289
0.9043	4.5	9216	1.7872	46.1526	62.7803	64.0856	68.4802

Framework versions

• Transformers 4.27.0
• Pytorch 2.0.0+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2