lora_alpha_64_drop_0.3_rank_32_seed_42_merges_40_08062024

This model is a fine-tuned version of xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4618
• Accuracy: 0.8305

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.3456	1.0	12272	0.4626	0.8289
0.3458	2.0	24544	0.4889	0.8253
0.3402	3.0	36816	0.4702	0.8285
0.3445	4.0	49088	0.4648	0.8333
0.3408	5.0	61360	0.4701	0.8301
0.341	6.0	73632	0.4628	0.8309
0.3275	7.0	85904	0.4700	0.8281
0.3455	8.0	98176	0.4572	0.8305
0.3455	9.0	110448	0.4795	0.8321
0.3336	10.0	122720	0.4597	0.8273
0.3317	11.0	134992	0.4716	0.8309
0.33	12.0	147264	0.4605	0.8341
0.3279	13.0	159536	0.4824	0.8265
0.323	14.0	171808	0.4634	0.8285
0.3276	15.0	184080	0.4876	0.8305
0.3254	16.0	196352	0.4658	0.8297
0.3349	17.0	208624	0.4713	0.8297
0.334	18.0	220896	0.4759	0.8285
0.3319	19.0	233168	0.4664	0.8321
0.3425	20.0	245440	0.4761	0.8285
0.3366	21.0	257712	0.4599	0.8329
0.3475	22.0	269984	0.4614	0.8313
0.3416	23.0	282256	0.4590	0.8317
0.3455	24.0	294528	0.4630	0.8305
0.3399	25.0	306800	0.4618	0.8305

Framework versions

• PEFT 0.11.1
• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.18.0
• Tokenizers 0.19.1