my_model

This model is a fine-tuned version of klue/roberta-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4546

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.5259	1.0	4820	1.3120
1.2116	2.0	9640	1.0790
1.0629	3.0	14460	0.9467
0.9722	4.0	19280	0.8650
0.912	5.0	24100	0.8258
0.8514	6.0	28920	0.7731
0.8308	7.0	33740	0.7558
0.7854	8.0	38560	0.7081
0.74	9.0	43380	0.6947
0.738	10.0	48200	0.6608
0.7335	11.0	53020	0.6485
0.675	12.0	57840	0.6354
0.6655	13.0	62660	0.6106
0.6458	14.0	67480	0.6029
0.6256	15.0	72300	0.5821
0.6191	16.0	77120	0.5737
0.5979	17.0	81940	0.5696
0.6	18.0	86760	0.5595
0.5812	19.0	91580	0.5317
0.5736	20.0	96400	0.5282
0.5597	21.0	101220	0.5407
0.5665	22.0	106040	0.5404
0.5546	23.0	110860	0.5320
0.5574	24.0	115680	0.5160
0.5346	25.0	120500	0.5220
0.5323	26.0	125320	0.5099
0.5251	27.0	130140	0.4943
0.5254	28.0	134960	0.4907
0.5154	29.0	139780	0.4761
0.4914	30.0	144600	0.4958
0.5085	31.0	149420	0.4595
0.4897	32.0	154240	0.4697
0.4868	33.0	159060	0.4664
0.4779	34.0	163880	0.4684
0.4732	35.0	168700	0.4781
0.4757	36.0	173520	0.4687
0.4702	37.0	178340	0.4484
0.4652	38.0	183160	0.4522
0.4522	39.0	187980	0.4622
0.4559	40.0	192800	0.4546
0.4558	41.0	197620	0.4370
0.4482	42.0	202440	0.4359
0.4451	43.0	207260	0.4467
0.4383	44.0	212080	0.4401
0.4489	45.0	216900	0.4309
0.4347	46.0	221720	0.4256
0.4356	47.0	226540	0.4423
0.4447	48.0	231360	0.4441
0.4324	49.0	236180	0.4405
0.4278	50.0	241000	0.4306

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0