XMLRobertaLexical-finetuned_70KURL_daydu

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1582
• Accuracy: 0.9693
• F1: 0.9693

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2326	200	0.1774	0.9352	0.9354
No log	0.4651	400	0.1531	0.9504	0.9507
No log	0.6977	600	0.1426	0.9565	0.9564
No log	0.9302	800	0.1279	0.9591	0.9593
0.2043	1.1628	1000	0.1533	0.9587	0.9591
0.2043	1.3953	1200	0.1247	0.9634	0.9636
0.2043	1.6279	1400	0.1245	0.9598	0.9602
0.2043	1.8605	1600	0.1141	0.9670	0.9671
0.1194	2.0930	1800	0.1187	0.9653	0.9655
0.1194	2.3256	2000	0.1453	0.9648	0.9648
0.1194	2.5581	2200	0.1248	0.9644	0.9644
0.1194	2.7907	2400	0.1039	0.9689	0.9690
0.0987	3.0233	2600	0.1243	0.9654	0.9654
0.0987	3.2558	2800	0.1133	0.9667	0.9669
0.0987	3.4884	3000	0.1146	0.9694	0.9695
0.0987	3.7209	3200	0.1073	0.9679	0.9681
0.0987	3.9535	3400	0.1099	0.9677	0.9679
0.0845	4.1860	3600	0.1115	0.9687	0.9688
0.0845	4.4186	3800	0.1150	0.9672	0.9674
0.0845	4.6512	4000	0.1165	0.9687	0.9688
0.0845	4.8837	4200	0.1152	0.9679	0.9681
0.071	5.1163	4400	0.1177	0.9685	0.9685
0.071	5.3488	4600	0.1257	0.9695	0.9696
0.071	5.5814	4800	0.1283	0.9681	0.9682
0.071	5.8140	5000	0.1418	0.9683	0.9684
0.0614	6.0465	5200	0.1359	0.9676	0.9676
0.0614	6.2791	5400	0.1417	0.9687	0.9687
0.0614	6.5116	5600	0.1294	0.9696	0.9697
0.0614	6.7442	5800	0.1376	0.9680	0.9680
0.0614	6.9767	6000	0.1328	0.9680	0.9681
0.0531	7.2093	6200	0.1349	0.9682	0.9683
0.0531	7.4419	6400	0.1475	0.9682	0.9683
0.0531	7.6744	6600	0.1425	0.9692	0.9693
0.0531	7.9070	6800	0.1401	0.9698	0.9699
0.047	8.1395	7000	0.1488	0.9686	0.9688
0.047	8.3721	7200	0.1406	0.9682	0.9683
0.047	8.6047	7400	0.1645	0.9685	0.9685
0.047	8.8372	7600	0.1427	0.9697	0.9698
0.0408	9.0698	7800	0.1465	0.9687	0.9688
0.0408	9.3023	8000	0.1475	0.9684	0.9684
0.0408	9.5349	8200	0.1533	0.9691	0.9691
0.0408	9.7674	8400	0.1573	0.9696	0.9696
0.0357	10.0	8600	0.1582	0.9693	0.9693

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1