deit-base-distilled-patch16-224-65-fold4

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5367
• Accuracy: 0.8451

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9231	3	0.7145	0.5211
No log	1.8462	6	0.7082	0.5070
No log	2.7692	9	0.6889	0.6056
0.6878	4.0	13	0.6703	0.6620
0.6878	4.9231	16	0.6556	0.6761
0.6878	5.8462	19	0.6430	0.6620
0.6203	6.7692	22	0.6250	0.6761
0.6203	8.0	26	0.7464	0.6197
0.6203	8.9231	29	0.6647	0.6056
0.5703	9.8462	32	0.6097	0.7042
0.5703	10.7692	35	0.6261	0.6620
0.5703	12.0	39	0.5926	0.7042
0.5281	12.9231	42	0.5370	0.7465
0.5281	13.8462	45	0.5638	0.7465
0.5281	14.7692	48	0.7175	0.6056
0.4616	16.0	52	0.8917	0.5775
0.4616	16.9231	55	0.6761	0.6761
0.4616	17.8462	58	0.5606	0.7324
0.4943	18.7692	61	0.6963	0.6338
0.4943	20.0	65	0.6462	0.6620
0.4943	20.9231	68	0.6246	0.7183
0.4058	21.8462	71	0.7336	0.6620
0.4058	22.7692	74	0.6270	0.7324
0.4058	24.0	78	0.6097	0.7183
0.3577	24.9231	81	0.6700	0.7606
0.3577	25.8462	84	0.6676	0.7183
0.3577	26.7692	87	0.5475	0.7887
0.2988	28.0	91	0.5383	0.8028
0.2988	28.9231	94	0.5534	0.7183
0.2988	29.8462	97	0.5842	0.8028
0.2595	30.7692	100	0.5965	0.7887
0.2595	32.0	104	0.6220	0.7606
0.2595	32.9231	107	0.6027	0.7606
0.2422	33.8462	110	0.6369	0.7183
0.2422	34.7692	113	0.6033	0.7746
0.2422	36.0	117	0.6912	0.7324
0.1927	36.9231	120	0.6582	0.7887
0.1927	37.8462	123	0.6320	0.7746
0.1927	38.7692	126	0.7532	0.7606
0.2399	40.0	130	0.7909	0.7606
0.2399	40.9231	133	0.6808	0.7465
0.2399	41.8462	136	0.5816	0.7887
0.2399	42.7692	139	0.5474	0.7887
0.2218	44.0	143	0.6310	0.7042
0.2218	44.9231	146	0.6453	0.8028
0.2218	45.8462	149	0.6170	0.7887
0.1817	46.7692	152	0.6034	0.7887
0.1817	48.0	156	0.6350	0.8310
0.1817	48.9231	159	0.6027	0.7887
0.1483	49.8462	162	0.5599	0.8028
0.1483	50.7692	165	0.5817	0.8310
0.1483	52.0	169	0.6086	0.7746
0.1668	52.9231	172	0.5744	0.8169
0.1668	53.8462	175	0.6059	0.7887
0.1668	54.7692	178	0.6455	0.7887
0.1372	56.0	182	0.5367	0.8451
0.1372	56.9231	185	0.5615	0.8169
0.1372	57.8462	188	0.6378	0.8028
0.1485	58.7692	191	0.5687	0.8169
0.1485	60.0	195	0.4897	0.8169
0.1485	60.9231	198	0.4384	0.8451
0.1426	61.8462	201	0.5087	0.7887
0.1426	62.7692	204	0.4757	0.8169
0.1426	64.0	208	0.4373	0.8169
0.1333	64.9231	211	0.4512	0.8169
0.1333	65.8462	214	0.4619	0.7887
0.1333	66.7692	217	0.5520	0.8028
0.1306	68.0	221	0.5161	0.7887
0.1306	68.9231	224	0.5180	0.7606
0.1306	69.8462	227	0.5778	0.8028
0.1327	70.7692	230	0.5933	0.8028
0.1327	72.0	234	0.5222	0.7887
0.1327	72.9231	237	0.5104	0.8169
0.1171	73.8462	240	0.5024	0.8169
0.1171	74.7692	243	0.5060	0.8028
0.1171	76.0	247	0.5267	0.7746
0.1227	76.9231	250	0.4775	0.8169
0.1227	77.8462	253	0.5020	0.8169
0.1227	78.7692	256	0.5243	0.7606
0.1304	80.0	260	0.6195	0.7887
0.1304	80.9231	263	0.5740	0.7606
0.1304	81.8462	266	0.5652	0.8169
0.1304	82.7692	269	0.5750	0.8169
0.1152	84.0	273	0.5829	0.7887
0.1152	84.9231	276	0.5854	0.7887
0.1152	85.8462	279	0.5854	0.7887
0.1069	86.7692	282	0.5826	0.8028
0.1069	88.0	286	0.5839	0.7887
0.1069	88.9231	289	0.5792	0.8169
0.122	89.8462	292	0.5755	0.8169
0.122	90.7692	295	0.5751	0.8169
0.122	92.0	299	0.5748	0.8169
0.1268	92.3077	300	0.5748	0.8169

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1