fine-tuned-DatasetQAS-Squad-ID-with-indobert-base-uncased-without-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6966
• Exact Match: 45.0886
• F1: 61.6033
• Precision: 63.1410
• Recall: 67.2829

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.06
• num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1	Precision	Recall
2.4337	0.5	1023	2.2704	30.4084	43.3793	45.1001	51.7593
1.8609	1.0	2046	1.8600	39.3745	56.3414	57.5512	63.3710
1.6152	1.5	3069	1.7283	41.8614	58.9008	60.4703	65.7497
1.5351	2.0	4092	1.6703	43.7661	61.0546	62.5984	66.7544
1.3952	2.5	5115	1.6566	44.0073	61.0769	62.7069	66.3452
1.3943	3.0	6138	1.6498	44.7309	62.0362	62.9790	68.2142
1.2659	3.5	7161	1.6441	44.8557	61.5081	62.8213	67.5871
1.2784	4.0	8184	1.6347	45.1302	61.9251	63.3021	67.6987
1.1424	4.5	9207	1.6966	45.0886	61.6033	63.1410	67.2829

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2