scenario-NON-KD-PR-COPY-CDF-CL-D2_data-cl-cardiff_cl_only_beta

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.2732
• Accuracy: 0.4498
• F1: 0.4463

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 112233
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.09	250	1.2151	0.4537	0.4483
0.9206	2.17	500	1.3326	0.4367	0.4368
0.9206	3.26	750	1.6141	0.4645	0.4596
0.5989	4.35	1000	1.8743	0.4452	0.4337
0.5989	5.43	1250	1.9653	0.4599	0.4556
0.3141	6.52	1500	2.4768	0.4429	0.4289
0.3141	7.61	1750	2.7079	0.4414	0.4409
0.1761	8.7	2000	2.6592	0.4545	0.4485
0.1761	9.78	2250	3.0138	0.4591	0.4596
0.1126	10.87	2500	3.5949	0.4460	0.4405
0.1126	11.96	2750	3.4857	0.4622	0.4626
0.0872	13.04	3000	3.8183	0.4460	0.4453
0.0872	14.13	3250	3.6123	0.4560	0.4508
0.0639	15.22	3500	4.0046	0.4545	0.4519
0.0639	16.3	3750	4.2095	0.4506	0.4481
0.0442	17.39	4000	4.6675	0.4329	0.4143
0.0442	18.48	4250	4.6690	0.4468	0.4415
0.0308	19.57	4500	4.4315	0.4630	0.4624
0.0308	20.65	4750	4.6030	0.4591	0.4579
0.0274	21.74	5000	4.7447	0.4576	0.4551
0.0274	22.83	5250	4.9064	0.4537	0.4511
0.0147	23.91	5500	5.2460	0.4437	0.4343
0.0147	25.0	5750	5.3218	0.4398	0.4331
0.0129	26.09	6000	5.0850	0.4491	0.4484
0.0129	27.17	6250	5.2391	0.4437	0.4398
0.0096	28.26	6500	5.2968	0.4468	0.4409
0.0096	29.35	6750	5.2732	0.4498	0.4463

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3