N_roberta_sst5_padding50model

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.9458
• Accuracy: 0.5321

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.5748	1.0	534	1.5892	0.2308
1.5742	2.0	1068	1.5757	0.3023
1.4713	3.0	1602	1.1312	0.4742
1.1254	4.0	2136	1.1239	0.5267
0.9762	5.0	2670	1.0890	0.5353
0.8536	6.0	3204	1.1455	0.5425
0.7226	7.0	3738	1.2372	0.5312
0.6262	8.0	4272	1.3701	0.5380
0.5263	9.0	4806	1.3611	0.5457
0.455	10.0	5340	1.5596	0.5290
0.3977	11.0	5874	1.7496	0.5158
0.3475	12.0	6408	1.8187	0.5276
0.2946	13.0	6942	2.1163	0.5312
0.2805	14.0	7476	2.3352	0.5290
0.2319	15.0	8010	2.5428	0.5253
0.2274	16.0	8544	2.6192	0.5208
0.1894	17.0	9078	2.7966	0.5267
0.1865	18.0	9612	2.8428	0.5326
0.1706	19.0	10146	2.9123	0.5317
0.142	20.0	10680	2.9458	0.5321

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.13.3