IndoBERT_top5_bm25_rr5_10_epoch

This model is a fine-tuned version of indolem/indobert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0484
• Accuracy: 0.8476
• F1: 0.7027
• Precision: 0.7143
• Recall: 0.6915

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
No log	0.2857	16	0.5745	0.7396	0.0	0.0	0.0
No log	0.5714	32	0.5547	0.7396	0.0	0.0	0.0
No log	0.8571	48	0.5288	0.7396	0.0	0.0	0.0
No log	1.1429	64	0.4822	0.8006	0.4462	0.8056	0.3085
No log	1.4286	80	0.4105	0.8310	0.6013	0.7797	0.4894
No log	1.7143	96	0.3975	0.8172	0.6633	0.6373	0.6915
No log	2.0	112	0.3980	0.8172	0.5541	0.7593	0.4362
No log	2.2857	128	0.4243	0.8144	0.6794	0.6174	0.7553
No log	2.5714	144	0.4404	0.8033	0.4580	0.8108	0.3191
No log	2.8571	160	0.3763	0.8504	0.6824	0.7632	0.6170
No log	3.1429	176	0.6084	0.7701	0.6527	0.5379	0.8298
No log	3.4286	192	0.4822	0.8587	0.7052	0.7722	0.6489
No log	3.7143	208	0.4620	0.8449	0.6164	0.8654	0.4787
No log	4.0	224	0.6729	0.7922	0.6809	0.5674	0.8511
No log	4.2857	240	0.7337	0.8449	0.7143	0.6863	0.7447
No log	4.5714	256	1.0946	0.7812	0.6580	0.5547	0.8085
No log	4.8571	272	1.0382	0.7535	0.6397	0.5163	0.8404
No log	5.1429	288	0.5228	0.8532	0.6971	0.7531	0.6489
No log	5.4286	304	0.8456	0.8255	0.6897	0.6422	0.7447
No log	5.7143	320	0.8758	0.8504	0.6860	0.7564	0.6277
No log	6.0	336	0.9307	0.8116	0.6699	0.6161	0.7340
No log	6.2857	352	0.7016	0.8421	0.6743	0.7284	0.6277
No log	6.5714	368	0.6991	0.8560	0.6941	0.7763	0.6277
No log	6.8571	384	0.7400	0.8504	0.7188	0.7041	0.7340
No log	7.1429	400	0.8463	0.8532	0.7166	0.7204	0.7128
No log	7.4286	416	0.8996	0.8560	0.7234	0.7234	0.7234
No log	7.7143	432	0.9267	0.8504	0.7158	0.7083	0.7234
No log	8.0	448	0.9227	0.8587	0.7182	0.7471	0.6915
No log	8.2857	464	0.9840	0.8476	0.7027	0.7143	0.6915
No log	8.5714	480	1.0115	0.8449	0.6923	0.7159	0.6702
No log	8.8571	496	1.0437	0.8449	0.6957	0.7111	0.6809
0.2421	9.1429	512	1.0514	0.8449	0.6957	0.7111	0.6809
0.2421	9.4286	528	1.0470	0.8476	0.7027	0.7143	0.6915
0.2421	9.7143	544	1.0438	0.8476	0.7027	0.7143	0.6915
0.2421	10.0	560	1.0484	0.8476	0.7027	0.7143	0.6915

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.1+cu121
• Tokenizers 0.19.1