fine-tuned-DatasetQAS-IDK-MRC-with-indobert-base-uncased-without-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0844
• Exact Match: 65.5759
• F1: 70.8360

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.4526	0.49	36	2.4554	49.8691	49.8691
3.7633	0.98	72	1.9974	49.7382	49.7438
2.2184	1.47	108	1.8592	49.7382	50.4880
2.2184	1.96	144	1.7572	49.3455	51.5995
2.0352	2.45	180	1.6433	49.3455	54.2883
1.8246	2.94	216	1.5385	54.0576	57.3719
1.6495	3.43	252	1.4355	57.9843	62.1664
1.6495	3.92	288	1.3803	57.7225	62.1946
1.5079	4.41	324	1.3151	57.4607	63.1030
1.3449	4.9	360	1.2581	59.2932	65.0643
1.3449	5.39	396	1.1867	62.1728	67.8726
1.2429	5.88	432	1.1721	63.0890	68.1874
1.1547	6.37	468	1.1433	64.2670	69.1473
1.0805	6.86	504	1.1245	64.5288	69.0480
1.0805	7.35	540	1.0896	65.8377	70.8688
1.0457	7.84	576	1.0936	66.0995	70.8063
1.0152	8.33	612	1.0979	65.9686	70.8546
1.0152	8.82	648	1.0924	65.8377	71.0837
0.9966	9.31	684	1.0849	65.4450	70.6378
0.965	9.8	720	1.0844	65.5759	70.8360

Framework versions

• Transformers 4.27.4
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2