fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2832
• Exact Match: 59.3368
• F1: 73.6394
• Precision: 75.6497
• Recall: 79.2494

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1	Precision	Recall
6.1305	0.49	38	2.9545	18.3246	28.7037	30.7234	39.3266
3.6666	0.99	76	2.0933	29.3194	41.5386	41.3158	57.3278
2.2221	1.48	114	1.5088	46.0733	59.6910	61.3465	70.0645
1.5513	1.97	152	1.2788	52.7051	67.6237	68.9352	76.7287
1.5513	2.47	190	1.2375	56.0209	70.0861	72.2276	76.3275
1.1584	2.96	228	1.1617	56.3700	70.9542	72.5147	77.8564
1.0032	3.45	266	1.1656	57.9407	72.1620	73.8214	78.2817
0.8661	3.95	304	1.1443	57.5916	72.5053	73.8808	80.3537
0.8661	4.44	342	1.1663	58.4642	73.4761	75.0381	80.0108
0.7541	4.94	380	1.1414	58.2897	73.1853	74.9363	78.6912
0.6687	5.43	418	1.2151	60.0349	73.6810	75.7886	79.3854
0.5926	5.92	456	1.1805	60.5585	74.6182	76.2757	81.1406
0.5926	6.42	494	1.2740	60.5585	74.4135	76.4582	80.1876
0.4761	6.91	532	1.2221	59.8604	74.5837	75.8985	80.5858
0.4644	7.4	570	1.2832	59.3368	73.6394	75.6497	79.2494

Framework versions

• Transformers 4.27.0
• Pytorch 2.0.0+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2