deit-base-distilled-patch16-224-65-fold5

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5156
• Accuracy: 0.8732

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9231	3	0.7425	0.4930
No log	1.8462	6	0.7193	0.5634
No log	2.7692	9	0.6808	0.5915
0.7309	4.0	13	0.6253	0.5915
0.7309	4.9231	16	0.6022	0.6761
0.7309	5.8462	19	0.5589	0.6479
0.6449	6.7692	22	0.5559	0.7183
0.6449	8.0	26	0.4910	0.7183
0.6449	8.9231	29	0.4996	0.7606
0.5494	9.8462	32	0.4903	0.7324
0.5494	10.7692	35	0.7331	0.6620
0.5494	12.0	39	0.5053	0.6901
0.4793	12.9231	42	0.4781	0.7324
0.4793	13.8462	45	0.4997	0.7465
0.4793	14.7692	48	0.5197	0.7465
0.4327	16.0	52	0.5339	0.7606
0.4327	16.9231	55	0.4475	0.7606
0.4327	17.8462	58	0.4808	0.7887
0.3747	18.7692	61	0.4868	0.7465
0.3747	20.0	65	0.6206	0.7042
0.3747	20.9231	68	0.5271	0.7324
0.3474	21.8462	71	0.5227	0.6901
0.3474	22.7692	74	0.5078	0.7465
0.3474	24.0	78	0.5842	0.6901
0.267	24.9231	81	0.6015	0.7183
0.267	25.8462	84	0.6533	0.7606
0.267	26.7692	87	0.5764	0.7324
0.2333	28.0	91	0.4862	0.8028
0.2333	28.9231	94	0.6233	0.7183
0.2333	29.8462	97	0.7549	0.7465
0.2635	30.7692	100	0.4890	0.8028
0.2635	32.0	104	0.5616	0.8028
0.2635	32.9231	107	0.5501	0.7606
0.192	33.8462	110	0.4845	0.8169
0.192	34.7692	113	0.5116	0.7887
0.192	36.0	117	0.5017	0.8169
0.1763	36.9231	120	0.4798	0.7887
0.1763	37.8462	123	0.5328	0.7746
0.1763	38.7692	126	0.6393	0.7606
0.172	40.0	130	0.5481	0.7887
0.172	40.9231	133	0.5867	0.7887
0.172	41.8462	136	0.9223	0.7042
0.172	42.7692	139	0.6262	0.8028
0.1832	44.0	143	0.6091	0.7746
0.1832	44.9231	146	0.5837	0.7606
0.1832	45.8462	149	0.5465	0.7606
0.1641	46.7692	152	0.6745	0.7746
0.1641	48.0	156	0.5398	0.7887
0.1641	48.9231	159	0.5387	0.8169
0.1366	49.8462	162	0.5737	0.8028
0.1366	50.7692	165	0.5255	0.8310
0.1366	52.0	169	0.6486	0.7887
0.149	52.9231	172	0.5404	0.8169
0.149	53.8462	175	0.5655	0.8169
0.149	54.7692	178	0.6121	0.8028
0.1196	56.0	182	0.6182	0.8310
0.1196	56.9231	185	0.6175	0.8028
0.1196	57.8462	188	0.5921	0.8310
0.1202	58.7692	191	0.5953	0.8169
0.1202	60.0	195	0.6065	0.8028
0.1202	60.9231	198	0.5448	0.8310
0.1289	61.8462	201	0.5258	0.8451
0.1289	62.7692	204	0.5440	0.8310
0.1289	64.0	208	0.6082	0.8169
0.1262	64.9231	211	0.6358	0.8169
0.1262	65.8462	214	0.5982	0.8169
0.1262	66.7692	217	0.5850	0.8451
0.124	68.0	221	0.5733	0.8169
0.124	68.9231	224	0.5631	0.8028
0.124	69.8462	227	0.5375	0.8310
0.1208	70.7692	230	0.5158	0.8169
0.1208	72.0	234	0.5431	0.8169
0.1208	72.9231	237	0.5099	0.8451
0.1126	73.8462	240	0.5803	0.7887
0.1126	74.7692	243	0.5416	0.8028
0.1126	76.0	247	0.5835	0.8451
0.1089	76.9231	250	0.5923	0.8310
0.1089	77.8462	253	0.5228	0.8310
0.1089	78.7692	256	0.5467	0.8310
0.0965	80.0	260	0.5156	0.8732
0.0965	80.9231	263	0.5082	0.8451
0.0965	81.8462	266	0.5071	0.8451
0.0965	82.7692	269	0.5070	0.8592
0.0947	84.0	273	0.5268	0.8592
0.0947	84.9231	276	0.5283	0.8592
0.0947	85.8462	279	0.5261	0.8451
0.0751	86.7692	282	0.5286	0.8310
0.0751	88.0	286	0.5415	0.8310
0.0751	88.9231	289	0.5511	0.8310
0.0912	89.8462	292	0.5542	0.8310
0.0912	90.7692	295	0.5464	0.8310
0.0912	92.0	299	0.5410	0.8310
0.104	92.3077	300	0.5407	0.8310

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1