fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-without-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7843
• Exact Match: 45.4462
• F1: 62.3862
• Precision: 63.7620
• Recall: 67.9874

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1	Precision	Recall
1.7808	0.5	1024	1.8192	39.1225	56.2210	57.3004	64.1431
1.598	1.0	2048	1.6753	42.0808	59.2251	60.4591	66.8518
1.4195	1.5	3072	1.6611	43.5599	60.7862	62.5640	66.8640
1.4262	2.0	4096	1.6420	45.0474	62.2214	63.9056	67.8747
1.1907	2.5	5120	1.6686	44.6319	61.3804	62.6910	68.0916
1.2017	3.0	6144	1.6597	45.4130	62.6561	63.9677	68.6391
1.0831	3.5	7168	1.7486	45.3714	62.2018	63.5154	68.2643
1.0256	4.0	8192	1.6899	45.6955	62.5503	64.2337	67.9408
0.9378	4.5	9216	1.7843	45.4462	62.3862	63.7620	67.9874

Framework versions

• Transformers 4.27.0
• Pytorch 2.0.0+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2