deit-base-distilled-patch16-224-55-fold4

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4908
• Accuracy: 0.8354

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.8571	3	0.8297	0.5063
No log	2.0	7	0.6548	0.6203
0.6826	2.8571	10	0.6513	0.5190
0.6826	4.0	14	0.6024	0.7342
0.6826	4.8571	17	0.6473	0.5823
0.6198	6.0	21	0.5717	0.6835
0.6198	6.8571	24	0.7837	0.5570
0.6198	8.0	28	0.5272	0.7595
0.5409	8.8571	31	0.5162	0.7342
0.5409	10.0	35	0.6156	0.6709
0.5409	10.8571	38	0.5844	0.6582
0.5177	12.0	42	0.7013	0.6709
0.5177	12.8571	45	0.5017	0.7722
0.5177	14.0	49	0.4723	0.7975
0.4995	14.8571	52	0.7057	0.6835
0.4995	16.0	56	0.4669	0.8228
0.4995	16.8571	59	0.5737	0.7468
0.3767	18.0	63	0.5035	0.8101
0.3767	18.8571	66	0.8993	0.6203
0.3602	20.0	70	0.5425	0.7975
0.3602	20.8571	73	0.5605	0.7722
0.3602	22.0	77	0.4260	0.7975
0.3479	22.8571	80	0.4117	0.8228
0.3479	24.0	84	0.4017	0.7975
0.3479	24.8571	87	0.4401	0.8228
0.2845	26.0	91	0.4490	0.7975
0.2845	26.8571	94	0.5080	0.8101
0.2845	28.0	98	0.5014	0.7975
0.2316	28.8571	101	0.5385	0.7722
0.2316	30.0	105	0.5643	0.7595
0.2316	30.8571	108	0.4887	0.7848
0.2212	32.0	112	0.5253	0.7975
0.2212	32.8571	115	0.4633	0.7975
0.2212	34.0	119	0.5079	0.8228
0.2093	34.8571	122	0.5082	0.7975
0.2093	36.0	126	0.6113	0.7468
0.2093	36.8571	129	0.5094	0.7848
0.179	38.0	133	0.6253	0.8101
0.179	38.8571	136	0.5406	0.7848
0.173	40.0	140	0.6687	0.7722
0.173	40.8571	143	0.5691	0.7975
0.173	42.0	147	0.5776	0.7975
0.1881	42.8571	150	0.7816	0.7595
0.1881	44.0	154	0.4908	0.8354
0.1881	44.8571	157	0.5330	0.7848
0.161	46.0	161	0.5351	0.7848
0.161	46.8571	164	0.6663	0.7848
0.161	48.0	168	0.5382	0.7975
0.1446	48.8571	171	0.5877	0.7848
0.1446	50.0	175	0.6856	0.8228
0.1446	50.8571	178	0.6044	0.8101
0.1556	52.0	182	0.7954	0.7848
0.1556	52.8571	185	0.7432	0.7848
0.1556	54.0	189	0.7896	0.8101
0.1602	54.8571	192	0.8400	0.8101
0.1602	56.0	196	0.8243	0.7848
0.1602	56.8571	199	0.6864	0.8354
0.1357	58.0	203	0.7131	0.7722
0.1357	58.8571	206	0.9191	0.7722
0.1262	60.0	210	0.7465	0.7722
0.1262	60.8571	213	0.7127	0.7848
0.1262	62.0	217	0.6973	0.7975
0.1323	62.8571	220	0.7125	0.7848
0.1323	64.0	224	0.7235	0.7848
0.1323	64.8571	227	0.7200	0.7975
0.1258	66.0	231	0.7616	0.7848
0.1258	66.8571	234	0.8537	0.7848
0.1258	68.0	238	0.8223	0.7848
0.1316	68.8571	241	0.7751	0.8228
0.1316	70.0	245	0.7689	0.8228
0.1316	70.8571	248	0.7751	0.8101
0.1104	72.0	252	0.7770	0.8101
0.1104	72.8571	255	0.7916	0.7975
0.1104	74.0	259	0.7995	0.7848
0.1239	74.8571	262	0.7981	0.7848
0.1239	76.0	266	0.7890	0.8228
0.1239	76.8571	269	0.7888	0.8228
0.1057	78.0	273	0.7957	0.8228
0.1057	78.8571	276	0.7979	0.8228
0.1015	80.0	280	0.7959	0.8101
0.1015	80.8571	283	0.7935	0.8101
0.1015	82.0	287	0.7973	0.8228
0.101	82.8571	290	0.8033	0.7975
0.101	84.0	294	0.8072	0.7975
0.101	84.8571	297	0.8049	0.7975
0.1116	85.7143	300	0.8042	0.8101

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1