deit-base-distilled-patch16-224-75-fold4

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.2564
• Accuracy: 0.9302

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.5613	0.6744
No log	2.0	4	0.6233	0.6977
No log	3.0	6	0.7506	0.6977
No log	4.0	8	0.6462	0.6977
0.5231	5.0	10	0.3889	0.8140
0.5231	6.0	12	0.3485	0.8372
0.5231	7.0	14	0.4327	0.8372
0.5231	8.0	16	0.4470	0.8372
0.5231	9.0	18	0.3067	0.8605
0.3699	10.0	20	0.3075	0.8605
0.3699	11.0	22	0.5433	0.8372
0.3699	12.0	24	0.5849	0.8140
0.3699	13.0	26	0.4509	0.8372
0.3699	14.0	28	0.5410	0.8372
0.2951	15.0	30	0.4710	0.8140
0.2951	16.0	32	0.3738	0.8605
0.2951	17.0	34	0.3704	0.8372
0.2951	18.0	36	0.2682	0.9070
0.2951	19.0	38	0.4187	0.8372
0.2279	20.0	40	0.3961	0.8372
0.2279	21.0	42	0.2976	0.8837
0.2279	22.0	44	0.5655	0.8140
0.2279	23.0	46	0.4931	0.8372
0.2279	24.0	48	0.2672	0.9070
0.1823	25.0	50	0.2795	0.9070
0.1823	26.0	52	0.3885	0.8605
0.1823	27.0	54	0.2779	0.9070
0.1823	28.0	56	0.2990	0.8605
0.1823	29.0	58	0.3090	0.8605
0.1314	30.0	60	0.2564	0.9302
0.1314	31.0	62	0.2825	0.8605
0.1314	32.0	64	0.3655	0.8605
0.1314	33.0	66	0.2486	0.9302
0.1314	34.0	68	0.2734	0.9070
0.1387	35.0	70	0.5242	0.8372
0.1387	36.0	72	0.5958	0.8372
0.1387	37.0	74	0.3472	0.8837
0.1387	38.0	76	0.3057	0.9070
0.1387	39.0	78	0.4018	0.8372
0.1266	40.0	80	0.3940	0.8372
0.1266	41.0	82	0.3568	0.8605
0.1266	42.0	84	0.3533	0.8837
0.1266	43.0	86	0.3451	0.8837
0.1266	44.0	88	0.3478	0.9070
0.1004	45.0	90	0.3195	0.9302
0.1004	46.0	92	0.3926	0.9070
0.1004	47.0	94	0.4169	0.8837
0.1004	48.0	96	0.4274	0.8837
0.1004	49.0	98	0.4061	0.9070
0.1033	50.0	100	0.4277	0.8605
0.1033	51.0	102	0.3977	0.9070
0.1033	52.0	104	0.4428	0.8605
0.1033	53.0	106	0.6753	0.8140
0.1033	54.0	108	0.6912	0.8140
0.0827	55.0	110	0.4201	0.8605
0.0827	56.0	112	0.3086	0.9302
0.0827	57.0	114	0.3150	0.9302
0.0827	58.0	116	0.4757	0.8605
0.0827	59.0	118	0.6409	0.8605
0.0889	60.0	120	0.5430	0.8837
0.0889	61.0	122	0.4044	0.8837
0.0889	62.0	124	0.3473	0.9302
0.0889	63.0	126	0.3485	0.9302
0.0889	64.0	128	0.3711	0.9302
0.088	65.0	130	0.4405	0.8837
0.088	66.0	132	0.6526	0.8605
0.088	67.0	134	0.7019	0.8605
0.088	68.0	136	0.5408	0.8605
0.088	69.0	138	0.4057	0.9302
0.0734	70.0	140	0.3797	0.9302
0.0734	71.0	142	0.3811	0.9302
0.0734	72.0	144	0.4040	0.9302
0.0734	73.0	146	0.4567	0.8837
0.0734	74.0	148	0.5161	0.9070
0.0721	75.0	150	0.5240	0.8837
0.0721	76.0	152	0.5048	0.9070
0.0721	77.0	154	0.4635	0.9070
0.0721	78.0	156	0.4510	0.9070
0.0721	79.0	158	0.4931	0.9070
0.0592	80.0	160	0.5368	0.8837
0.0592	81.0	162	0.5297	0.8837
0.0592	82.0	164	0.4722	0.9070
0.0592	83.0	166	0.4179	0.9302
0.0592	84.0	168	0.4045	0.9302
0.0634	85.0	170	0.4200	0.9302
0.0634	86.0	172	0.4497	0.9302
0.0634	87.0	174	0.4796	0.9070
0.0634	88.0	176	0.4997	0.8837
0.0634	89.0	178	0.4867	0.8837
0.0758	90.0	180	0.4478	0.9302
0.0758	91.0	182	0.4145	0.9302
0.0758	92.0	184	0.4036	0.9302
0.0758	93.0	186	0.3877	0.9302
0.0758	94.0	188	0.3747	0.9070
0.0529	95.0	190	0.3685	0.9070
0.0529	96.0	192	0.3644	0.9070
0.0529	97.0	194	0.3648	0.9070
0.0529	98.0	196	0.3663	0.9070
0.0529	99.0	198	0.3686	0.9070
0.0594	100.0	200	0.3693	0.9070

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1