results

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6970

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.4077	0.0782	10000	1.2127
1.3837	0.1565	20000	1.1769
1.264	0.2347	30000	1.1395
1.2336	0.3130	40000	1.2410
1.1166	0.3912	50000	1.1913
1.2681	0.4695	60000	1.1694
1.1209	0.5477	70000	1.1809
1.1211	0.6259	80000	1.1599
1.0544	0.7042	90000	1.1876
1.1172	0.7824	100000	1.1239
0.9684	0.8607	110000	1.1358
1.0254	0.9389	120000	1.1637
1.0335	1.0171	130000	1.1263
1.0661	1.0954	140000	1.1546
0.9775	1.1736	150000	1.0895
0.9167	1.2519	160000	0.8886
0.8994	1.3301	170000	0.8791
0.9155	1.4084	180000	1.0548
0.9194	1.4866	190000	1.1017
0.8942	1.5648	200000	1.0410
0.865	1.6431	210000	1.0301
0.9295	1.7213	220000	nan
0.9483	1.7996	230000	1.0066
0.8371	1.8778	240000	1.0697
0.8137	1.9560	250000	1.0518
0.8199	2.0343	260000	0.9879
0.774	2.1125	270000	1.0009
0.8802	2.1908	280000	1.0175
0.7443	2.2690	290000	1.0213
0.7648	2.3473	300000	0.9495
0.8057	2.4255	310000	0.9668
0.7384	2.5037	320000	0.9304
0.7542	2.5820	330000	0.9320
0.7462	2.6602	340000	0.9513
0.7154	2.7385	350000	0.9557
0.7571	2.8167	360000	0.9425
0.7021	2.8949	370000	0.9010
0.7114	2.9732	380000	0.9036

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0