deit-base-distilled-patch16-224-65-fold2

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3095
• Accuracy: 0.9155

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9231	3	0.7211	0.4366
No log	1.8462	6	0.7016	0.5070
No log	2.7692	9	0.6703	0.6197
0.6946	4.0	13	0.6381	0.6620
0.6946	4.9231	16	0.5945	0.6761
0.6946	5.8462	19	0.6084	0.7183
0.6262	6.7692	22	0.5639	0.7465
0.6262	8.0	26	0.5203	0.7746
0.6262	8.9231	29	0.4805	0.7887
0.544	9.8462	32	0.5204	0.7324
0.544	10.7692	35	0.4635	0.7746
0.544	12.0	39	0.4957	0.7606
0.516	12.9231	42	0.4723	0.7746
0.516	13.8462	45	0.5170	0.7042
0.516	14.7692	48	0.5405	0.8169
0.4938	16.0	52	0.5082	0.7324
0.4938	16.9231	55	0.4608	0.7887
0.4938	17.8462	58	0.4211	0.7606
0.4123	18.7692	61	0.5015	0.7746
0.4123	20.0	65	0.3935	0.8592
0.4123	20.9231	68	0.4179	0.8169
0.3489	21.8462	71	0.3991	0.9014
0.3489	22.7692	74	0.3910	0.8592
0.3489	24.0	78	0.4277	0.8310
0.2889	24.9231	81	0.4032	0.8169
0.2889	25.8462	84	0.3703	0.8592
0.2889	26.7692	87	0.4404	0.8310
0.2659	28.0	91	0.3666	0.8592
0.2659	28.9231	94	0.3992	0.8169
0.2659	29.8462	97	0.4040	0.8169
0.2269	30.7692	100	0.3559	0.8592
0.2269	32.0	104	0.4766	0.8028
0.2269	32.9231	107	0.3852	0.8592
0.2031	33.8462	110	0.3702	0.8592
0.2031	34.7692	113	0.3203	0.8732
0.2031	36.0	117	0.5303	0.8169
0.2037	36.9231	120	0.3897	0.8732
0.2037	37.8462	123	0.3841	0.8732
0.2037	38.7692	126	0.3896	0.8873
0.2018	40.0	130	0.4177	0.8451
0.2018	40.9231	133	0.4548	0.8451
0.2018	41.8462	136	0.4115	0.8310
0.2018	42.7692	139	0.4121	0.8451
0.1721	44.0	143	0.3920	0.8592
0.1721	44.9231	146	0.3693	0.8451
0.1721	45.8462	149	0.3605	0.8592
0.1678	46.7692	152	0.5434	0.8310
0.1678	48.0	156	0.4189	0.8310
0.1678	48.9231	159	0.3124	0.8873
0.1604	49.8462	162	0.3293	0.8873
0.1604	50.7692	165	0.3372	0.9014
0.1604	52.0	169	0.3505	0.8732
0.1406	52.9231	172	0.3095	0.9155
0.1406	53.8462	175	0.3054	0.9155
0.1406	54.7692	178	0.3695	0.8873
0.1492	56.0	182	0.4058	0.8592
0.1492	56.9231	185	0.4650	0.8451
0.1492	57.8462	188	0.4060	0.8592
0.1359	58.7692	191	0.3819	0.8873
0.1359	60.0	195	0.5230	0.7887
0.1359	60.9231	198	0.4986	0.8169
0.1264	61.8462	201	0.4570	0.8310
0.1264	62.7692	204	0.4507	0.8451
0.1264	64.0	208	0.5765	0.8028
0.1478	64.9231	211	0.4514	0.8592
0.1478	65.8462	214	0.4434	0.8873
0.1478	66.7692	217	0.4403	0.8592
0.1398	68.0	221	0.5928	0.8310
0.1398	68.9231	224	0.4587	0.8592
0.1398	69.8462	227	0.4053	0.8451
0.161	70.7692	230	0.4233	0.8592
0.161	72.0	234	0.4264	0.8592
0.161	72.9231	237	0.4127	0.8310
0.1326	73.8462	240	0.4013	0.8592
0.1326	74.7692	243	0.4389	0.8451
0.1326	76.0	247	0.3772	0.8592
0.1236	76.9231	250	0.3600	0.8732
0.1236	77.8462	253	0.3890	0.8873
0.1236	78.7692	256	0.4401	0.8451
0.0973	80.0	260	0.4014	0.8592
0.0973	80.9231	263	0.3766	0.8732
0.0973	81.8462	266	0.3908	0.8451
0.0973	82.7692	269	0.4339	0.8592
0.1079	84.0	273	0.4567	0.8592
0.1079	84.9231	276	0.4415	0.8732
0.1079	85.8462	279	0.4183	0.8592
0.1015	86.7692	282	0.4039	0.8873
0.1015	88.0	286	0.3996	0.8873
0.1015	88.9231	289	0.4031	0.9014
0.1174	89.8462	292	0.4101	0.8732
0.1174	90.7692	295	0.4153	0.8732
0.1174	92.0	299	0.4146	0.8732
0.0968	92.3077	300	0.4145	0.8732

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1