fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-base-uncased-without-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2532
• Exact Match: 54.3210
• F1: 68.9707

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.5175	0.5	19	3.9501	7.5838	20.4343
6.5175	1.0	38	2.9055	20.1058	31.7904
4.0122	1.5	57	2.4641	20.8113	34.2803
4.0122	2.0	76	2.2939	26.1023	39.8860
4.0122	2.5	95	2.1319	28.9242	40.9501
2.4422	3.0	114	2.0104	32.8042	45.3945
2.4422	3.5	133	1.9276	34.9206	47.5124
2.0745	4.0	152	1.7986	37.5661	50.2540
2.0745	4.5	171	1.7248	41.6226	55.5665
2.0745	5.0	190	1.6329	43.9153	57.2818
1.7428	5.5	209	1.5379	46.7372	61.1075
1.7428	6.0	228	1.4441	49.9118	63.9580
1.7428	6.5	247	1.3759	51.8519	65.6742
1.4867	7.0	266	1.3336	52.5573	66.5658
1.4867	7.5	285	1.3048	53.0864	67.1039
1.3148	8.0	304	1.2778	54.3210	69.0039
1.3148	8.5	323	1.2593	53.9683	68.5121
1.3148	9.0	342	1.2590	53.6155	68.3475
1.2395	9.5	361	1.2532	54.3210	68.9707

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2