20230928-2-xlm-roberta-base-new

This model is a fine-tuned version of xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Accuracy: 0.4964
• Loss: 2.7563

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Accuracy	Validation Loss
4.4132	0.46	200	0.2795	nan
4.0135	0.91	400	0.3245	nan
3.874	1.37	600	0.2875	nan
3.6614	1.82	800	0.3380	3.4541
3.5348	2.28	1000	0.3618	3.2732
3.4756	2.73	1200	0.3986	nan
3.3677	3.19	1400	0.4245	nan
3.3707	3.64	1600	0.4044	3.3262
3.1909	4.1	1800	0.3968	nan
3.1404	4.56	2000	0.4360	3.2661
2.9553	5.01	2200	0.4752	2.7995
2.9725	5.47	2400	0.4255	2.9909
2.9121	5.92	2600	0.4724	2.7879
2.8641	6.38	2800	0.4727	nan
2.7376	6.83	3000	0.4414	2.9275
2.8078	7.29	3200	0.4766	2.5626
2.8166	7.74	3400	0.48	nan
2.6979	8.2	3600	0.5013	nan
2.7525	8.66	3800	0.4915	2.8394
2.6757	9.11	4000	0.5013	nan
2.6633	9.57	4200	0.4964	2.7563

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3