incorrect-order-distilbert-base-uncased

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1266
• Accuracy: 0.9857

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 70
• eval_batch_size: 70
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	37	0.1512	0.9672
No log	2.0	74	0.0947	0.9795
No log	3.0	111	0.0930	0.9795
No log	4.0	148	0.1328	0.9713
No log	5.0	185	0.1084	0.9816
No log	6.0	222	0.0798	0.9877
No log	7.0	259	0.1142	0.9775
No log	8.0	296	0.1182	0.9816
No log	9.0	333	0.1213	0.9816
No log	10.0	370	0.1227	0.9836
No log	11.0	407	0.1299	0.9816
No log	12.0	444	0.0980	0.9877
No log	13.0	481	0.1156	0.9836
0.0417	14.0	518	0.1440	0.9775
0.0417	15.0	555	0.1504	0.9775
0.0417	16.0	592	0.1337	0.9775
0.0417	17.0	629	0.1342	0.9775
0.0417	18.0	666	0.1191	0.9857
0.0417	19.0	703	0.1198	0.9857
0.0417	20.0	740	0.1210	0.9857
0.0417	21.0	777	0.1222	0.9857
0.0417	22.0	814	0.1230	0.9857
0.0417	23.0	851	0.1236	0.9857
0.0417	24.0	888	0.1242	0.9857
0.0417	25.0	925	0.1246	0.9857
0.0417	26.0	962	0.1251	0.9857
0.0417	27.0	999	0.1258	0.9857
0.0008	28.0	1036	0.1264	0.9857
0.0008	29.0	1073	0.1266	0.9857
0.0008	30.0	1110	0.1266	0.9857

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.0
• Datasets 2.20.0
• Tokenizers 0.19.1