lora_alpha_64_drop_0.3_rank_32_seed_42_merges_40_07062024

This model is a fine-tuned version of xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5089
• Accuracy: 0.8285

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.5585	1.0	12272	0.4933	0.8068
0.5257	2.0	24544	0.5033	0.8088
0.5008	3.0	36816	0.5443	0.7863
0.508	4.0	49088	0.4915	0.8088
0.4835	5.0	61360	0.4870	0.8197
0.4698	6.0	73632	0.4990	0.8145
0.4544	7.0	85904	0.4966	0.8112
0.4616	8.0	98176	0.5024	0.8133
0.4465	9.0	110448	0.4894	0.8273
0.4277	10.0	122720	0.4908	0.8225
0.4152	11.0	134992	0.5215	0.8076
0.4012	12.0	147264	0.4638	0.8325
0.3986	13.0	159536	0.5082	0.8209
0.3744	14.0	171808	0.4727	0.8261
0.3667	15.0	184080	0.5049	0.8249
0.3565	16.0	196352	0.5039	0.8181
0.3632	17.0	208624	0.5131	0.8249
0.3381	18.0	220896	0.4964	0.8265
0.3205	19.0	233168	0.5074	0.8273
0.3199	20.0	245440	0.5089	0.8285

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.18.0
• Tokenizers 0.19.1