scenario-KD-PR-CDF-EN-FROM-EN-D2_data-en-cardiff_eng_only_delta-jason

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 25.8673
• Accuracy: 0.3959
• F1: 0.3838

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 7777
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.72	100	21.4869	0.3399	0.2229
No log	3.45	200	21.6387	0.3699	0.3103
No log	5.17	300	21.3511	0.3907	0.3762
No log	6.9	400	21.9513	0.3968	0.3590
22.0328	8.62	500	21.5760	0.4048	0.3925
22.0328	10.34	600	21.8280	0.4259	0.4236
22.0328	12.07	700	22.1319	0.4096	0.4040
22.0328	13.79	800	23.1465	0.3884	0.3602
22.0328	15.52	900	23.6087	0.3907	0.3658
15.8082	17.24	1000	24.1019	0.3968	0.3767
15.8082	18.97	1100	24.2550	0.3973	0.3850
15.8082	20.69	1200	23.9667	0.4092	0.4043
15.8082	22.41	1300	25.2656	0.4145	0.4010
15.8082	24.14	1400	26.0200	0.3893	0.3638
11.3074	25.86	1500	25.2350	0.4101	0.3887
11.3074	27.59	1600	25.8133	0.4012	0.3853
11.3074	29.31	1700	25.8673	0.3959	0.3838

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3