xlm-roberta-xl-final-lora152520

This model is a fine-tuned version of facebook/xlm-roberta-xl on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5503
• Precision: 0.9267
• Recall: 0.9291
• F1: 0.9279
• Accuracy: 0.9386

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 40
• num_epochs: 40
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.2

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
2.6885	1.0	250	1.9699	0.7944	0.8343	0.8139	0.8442
1.7948	2.0	500	1.6854	0.8702	0.8791	0.8746	0.8949
1.6148	3.0	750	1.6185	0.8827	0.8998	0.8911	0.9096
1.5365	4.0	1000	1.5710	0.9031	0.9054	0.9043	0.9195
1.4852	5.0	1250	1.5524	0.9124	0.9129	0.9126	0.9255
1.4538	6.0	1500	1.5431	0.9112	0.9176	0.9144	0.9272
1.4306	7.0	1750	1.5390	0.9145	0.9221	0.9183	0.9297
1.4132	8.0	2000	1.5358	0.9191	0.9219	0.9205	0.9321
1.4004	9.0	2250	1.5365	0.9174	0.9262	0.9218	0.9337
1.3883	10.0	2500	1.5407	0.9176	0.9263	0.9220	0.9332
1.3803	11.0	2750	1.5326	0.9218	0.9278	0.9248	0.9358
1.3727	12.0	3000	1.5353	0.9187	0.9245	0.9216	0.9329
1.3674	13.0	3250	1.5392	0.9202	0.9254	0.9228	0.9350
1.3609	14.0	3500	1.5384	0.9220	0.9259	0.9239	0.9347
1.3572	15.0	3750	1.5382	0.9201	0.9240	0.9220	0.9334
1.3522	16.0	4000	1.5410	0.9197	0.9270	0.9233	0.9342
1.3502	17.0	4250	1.5449	0.9245	0.9268	0.9256	0.9355
1.3456	18.0	4500	1.5439	0.9233	0.9278	0.9256	0.9360
1.3423	19.0	4750	1.5435	0.9259	0.9248	0.9253	0.9346
1.3419	20.0	5000	1.5432	0.9270	0.9282	0.9276	0.9371
1.3397	21.0	5250	1.5398	0.9250	0.9284	0.9267	0.9369
1.3377	22.0	5500	1.5411	0.9253	0.9270	0.9262	0.9358
1.3351	23.0	5750	1.5471	0.9274	0.9284	0.9279	0.9374
1.3369	24.0	6000	1.5542	0.9214	0.9240	0.9227	0.9339
1.3348	25.0	6250	1.5479	0.9268	0.9288	0.9278	0.9374
1.334	26.0	6500	1.5492	0.9268	0.9294	0.9281	0.9384
1.3334	27.0	6750	1.5471	0.9299	0.9287	0.9293	0.9377
1.3327	28.0	7000	1.5438	0.9291	0.9309	0.9300	0.9394
1.3314	29.0	7250	1.5445	0.9304	0.9315	0.9310	0.9403
1.3318	30.0	7500	1.5456	0.9291	0.9310	0.9301	0.9399
1.3312	31.0	7750	1.5474	0.9278	0.9295	0.9287	0.9386
1.3304	32.0	8000	1.5489	0.9273	0.9302	0.9288	0.9388
1.3298	33.0	8250	1.5469	0.9286	0.9299	0.9293	0.9388
1.3295	34.0	8500	1.5474	0.9291	0.9312	0.9302	0.9398
1.3288	35.0	8750	1.5518	0.9280	0.9300	0.9290	0.9386
1.3292	36.0	9000	1.5484	0.9271	0.9308	0.9289	0.9388
1.3287	37.0	9250	1.5487	0.9278	0.9297	0.9287	0.9382
1.328	38.0	9500	1.5492	0.9290	0.9305	0.9298	0.9394
1.3281	39.0	9750	1.5496	0.9278	0.9293	0.9285	0.9387
1.3285	40.0	10000	1.5503	0.9267	0.9291	0.9279	0.9386

Framework versions

• Transformers 4.35.2
• Pytorch 2.0.1+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0