XMLRoberta_Dataset59KBoDuoi

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4792
• Accuracy: 0.8964
• F1: 0.8969

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.5115	200	0.4025	0.8084	0.8111
No log	1.0230	400	0.3500	0.8424	0.8451
No log	1.5345	600	0.3312	0.8637	0.8612
0.4018	2.0460	800	0.3394	0.8580	0.8610
0.4018	2.5575	1000	0.2938	0.8747	0.8760
0.4018	3.0691	1200	0.2903	0.8829	0.8841
0.4018	3.5806	1400	0.2871	0.8854	0.8859
0.2576	4.0921	1600	0.2955	0.8864	0.8873
0.2576	4.6036	1800	0.2831	0.8887	0.8894
0.2576	5.1151	2000	0.2952	0.8885	0.8898
0.2576	5.6266	2200	0.2947	0.8872	0.8881
0.2036	6.1381	2400	0.3086	0.8887	0.8902
0.2036	6.6496	2600	0.2939	0.8924	0.8931
0.2036	7.1611	2800	0.3368	0.8879	0.8895
0.2036	7.6726	3000	0.3162	0.8924	0.8932
0.1616	8.1841	3200	0.3423	0.8909	0.8919
0.1616	8.6957	3400	0.3475	0.8940	0.8945
0.1616	9.2072	3600	0.3546	0.8914	0.8923
0.1616	9.7187	3800	0.3505	0.8941	0.8947
0.1291	10.2302	4000	0.3850	0.8934	0.8941
0.1291	10.7417	4200	0.3718	0.8957	0.8963
0.1291	11.2532	4400	0.3893	0.8916	0.8924
0.1291	11.7647	4600	0.3923	0.8949	0.8955
0.1047	12.2762	4800	0.4213	0.8959	0.8968
0.1047	12.7877	5000	0.3877	0.8951	0.8961
0.1047	13.2992	5200	0.3972	0.8990	0.8992
0.1047	13.8107	5400	0.3896	0.8928	0.8937
0.0865	14.3223	5600	0.4290	0.8961	0.8964
0.0865	14.8338	5800	0.4360	0.8977	0.8979
0.0865	15.3453	6000	0.4398	0.8958	0.8963
0.0865	15.8568	6200	0.4357	0.8951	0.8955
0.0726	16.3683	6400	0.4662	0.8952	0.8953
0.0726	16.8798	6600	0.4608	0.8945	0.8955
0.0726	17.3913	6800	0.4714	0.8952	0.8954
0.0726	17.9028	7000	0.4638	0.8967	0.8971
0.0612	18.4143	7200	0.4783	0.8969	0.8971
0.0612	18.9258	7400	0.4856	0.8962	0.8967
0.0612	19.4373	7600	0.4779	0.8958	0.8963
0.0612	19.9488	7800	0.4792	0.8964	0.8969

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1