Revision_XLMRoBERTa_Lexical_Dataset_46k

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5120
• Accuracy: 0.8839
• F1: 0.8740

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2770	200	0.4447	0.7793	0.7705
No log	0.5540	400	0.3709	0.8294	0.8194
No log	0.8310	600	0.3417	0.8501	0.8302
0.4481	1.1080	800	0.3122	0.8596	0.8507
0.4481	1.3850	1000	0.3152	0.8653	0.8567
0.4481	1.6620	1200	0.2885	0.8792	0.8699
0.4481	1.9391	1400	0.2748	0.8840	0.8732
0.3197	2.2161	1600	0.3032	0.8815	0.8707
0.3197	2.4931	1800	0.2796	0.8800	0.8701
0.3197	2.7701	2000	0.2692	0.8875	0.8772
0.2652	3.0471	2200	0.2756	0.8833	0.8753
0.2652	3.3241	2400	0.2627	0.8879	0.8797
0.2652	3.6011	2600	0.2578	0.8882	0.8793
0.2652	3.8781	2800	0.2646	0.8841	0.8771
0.2335	4.1551	3000	0.3055	0.8843	0.8741
0.2335	4.4321	3200	0.2817	0.8890	0.8804
0.2335	4.7091	3400	0.2753	0.8839	0.8751
0.2335	4.9861	3600	0.2616	0.8855	0.8778
0.2087	5.2632	3800	0.2931	0.8899	0.8801
0.2087	5.5402	4000	0.2685	0.8883	0.8784
0.2087	5.8172	4200	0.3115	0.8859	0.8765
0.1855	6.0942	4400	0.3158	0.8870	0.8780
0.1855	6.3712	4600	0.3256	0.8839	0.8714
0.1855	6.6482	4800	0.3169	0.8914	0.8805
0.1855	6.9252	5000	0.3250	0.8869	0.8773
0.1632	7.2022	5200	0.3493	0.8866	0.8763
0.1632	7.4792	5400	0.3238	0.8825	0.8741
0.1632	7.7562	5600	0.3481	0.8850	0.8748
0.1414	8.0332	5800	0.3643	0.8875	0.8763
0.1414	8.3102	6000	0.4016	0.8856	0.8762
0.1414	8.5873	6200	0.3555	0.8901	0.8796
0.1414	8.8643	6400	0.3815	0.8860	0.8752
0.1256	9.1413	6600	0.3854	0.8859	0.8763
0.1256	9.4183	6800	0.3613	0.8910	0.8801
0.1256	9.6953	7000	0.3740	0.8898	0.8810
0.1256	9.9723	7200	0.3773	0.8872	0.8782
0.1092	10.2493	7400	0.4253	0.8830	0.8737
0.1092	10.5263	7600	0.4031	0.8820	0.8726
0.1092	10.8033	7800	0.4109	0.8861	0.8764
0.0978	11.0803	8000	0.4433	0.8835	0.8716
0.0978	11.3573	8200	0.4139	0.8841	0.8750
0.0978	11.6343	8400	0.4535	0.8850	0.8749
0.0978	11.9114	8600	0.4284	0.8845	0.8750
0.0877	12.1884	8800	0.4654	0.8840	0.8738
0.0877	12.4654	9000	0.4734	0.8838	0.8739
0.0877	12.7424	9200	0.4535	0.8855	0.8758
0.0781	13.0194	9400	0.4789	0.8833	0.8743
0.0781	13.2964	9600	0.4720	0.8850	0.8753
0.0781	13.5734	9800	0.4984	0.8856	0.8755
0.0781	13.8504	10000	0.4827	0.8848	0.8744
0.0698	14.1274	10200	0.5120	0.8833	0.8734
0.0698	14.4044	10400	0.5055	0.8845	0.8749
0.0698	14.6814	10600	0.5122	0.8839	0.8736
0.0698	14.9584	10800	0.5120	0.8839	0.8740

Framework versions

• Transformers 5.3.0
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.2