commitRoBertaWhy

This model is a fine-tuned version of xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7709
• Accuracy: 0.7818

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	483	0.4784	0.7515
0.5809	2.0	966	0.7377	0.7636
0.5228	3.0	1449	0.9487	0.7879
0.5178	4.0	1932	0.7810	0.8061
0.4931	5.0	2415	0.8674	0.8182
0.4562	6.0	2898	0.8308	0.8121
0.4585	7.0	3381	1.2448	0.7818
0.3061	8.0	3864	1.0845	0.8061
0.3341	9.0	4347	1.1429	0.7879
0.288	10.0	4830	1.0559	0.8242
0.2673	11.0	5313	1.2845	0.8
0.1954	12.0	5796	1.4496	0.8
0.227	13.0	6279	1.3763	0.8
0.1761	14.0	6762	1.4341	0.8061
0.2154	15.0	7245	1.3411	0.8061
0.1736	16.0	7728	1.4826	0.7879
0.0983	17.0	8211	1.7190	0.7818
0.1382	18.0	8694	1.5242	0.8
0.1262	19.0	9177	1.6978	0.7879
0.0962	20.0	9660	1.7709	0.7818

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3