distilbert_sst5_padding40model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.1102
• Accuracy: 0.5032

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.2973	1.0	534	1.2639	0.4226
1.0311	2.0	1068	1.1147	0.5122
0.8351	3.0	1602	1.2030	0.5086
0.68	4.0	2136	1.3100	0.5213
0.5243	5.0	2670	1.5365	0.4995
0.3959	6.0	3204	1.7658	0.4946
0.304	7.0	3738	1.9953	0.5050
0.2223	8.0	4272	2.1893	0.4991
0.1758	9.0	4806	2.5567	0.4986
0.1431	10.0	5340	2.8571	0.4982
0.1208	11.0	5874	3.0516	0.5027
0.1104	12.0	6408	3.3050	0.5095
0.0792	13.0	6942	3.5054	0.5014
0.0761	14.0	7476	3.6607	0.5009
0.0447	15.0	8010	3.8772	0.5032
0.0377	16.0	8544	3.9335	0.5050
0.0297	17.0	9078	4.0339	0.5059
0.0255	18.0	9612	4.0308	0.5095
0.0258	19.0	10146	4.0368	0.5086
0.0182	20.0	10680	4.1102	0.5032

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.13.3