distilbert_sst5_padding100model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.6964
• Accuracy: 0.4878

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.5744	1.0	534	1.6004	0.2308
1.4231	2.0	1068	1.2088	0.4729
1.1237	3.0	1602	1.1682	0.5
0.9507	4.0	2136	1.2027	0.5054
0.7604	5.0	2670	1.3283	0.4995
0.6266	6.0	3204	1.4933	0.4959
0.488	7.0	3738	1.6948	0.4851
0.3806	8.0	4272	1.8964	0.4896
0.3127	9.0	4806	1.9536	0.5014
0.2609	10.0	5340	2.1723	0.4919
0.2133	11.0	5874	2.4683	0.4864
0.1876	12.0	6408	2.6453	0.4941
0.1634	13.0	6942	2.9011	0.4891
0.1386	14.0	7476	3.0697	0.4941
0.1026	15.0	8010	3.3209	0.4900
0.0909	16.0	8544	3.5261	0.4914
0.0728	17.0	9078	3.5774	0.4873
0.0756	18.0	9612	3.6430	0.4891
0.059	19.0	10146	3.6841	0.4873
0.0476	20.0	10680	3.6964	0.4878

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.13.3