scenario-KD-PR-MSV-D2_data-cl-cardiff_cl_only_beta-jason

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 15.7103
• Accuracy: 0.4028
• F1: 0.4028

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 6666
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.09	250	11.8668	0.3897	0.3820
13.776	2.17	500	11.3224	0.3873	0.3811
13.776	3.26	750	10.7331	0.3881	0.3824
11.4609	4.35	1000	11.7060	0.4035	0.4015
11.4609	5.43	1250	11.1579	0.3997	0.3802
9.9802	6.52	1500	11.6003	0.4066	0.4061
9.9802	7.61	1750	11.6088	0.4059	0.4048
8.6874	8.7	2000	11.9784	0.3904	0.3814
8.6874	9.78	2250	12.2923	0.4113	0.4097
7.5941	10.87	2500	13.1464	0.3858	0.3823
7.5941	11.96	2750	12.8350	0.3966	0.3946
6.5229	13.04	3000	13.1611	0.3850	0.3819
6.5229	14.13	3250	14.1517	0.4005	0.3995
5.6501	15.22	3500	14.0929	0.4005	0.3930
5.6501	16.3	3750	14.1956	0.4074	0.4070
4.9968	17.39	4000	13.8417	0.4043	0.4040
4.9968	18.48	4250	14.3873	0.3897	0.3879
4.4769	19.57	4500	15.4822	0.4244	0.4226
4.4769	20.65	4750	15.1566	0.3958	0.3952
3.9676	21.74	5000	14.8283	0.4159	0.4135
3.9676	22.83	5250	15.2368	0.3927	0.3928
3.6886	23.91	5500	15.4609	0.4005	0.4006
3.6886	25.0	5750	14.7384	0.4059	0.4038
3.4119	26.09	6000	15.4645	0.3858	0.3857
3.4119	27.17	6250	15.9168	0.3974	0.3967
3.1245	28.26	6500	15.4980	0.3920	0.3923
3.1245	29.35	6750	15.7103	0.4028	0.4028

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3