deit-base-distilled-patch16-224-55-fold5

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.6452
• Accuracy: 0.8228

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.8571	3	0.7319	0.5190
No log	2.0	7	0.6756	0.5696
0.6544	2.8571	10	0.6199	0.6456
0.6544	4.0	14	0.5987	0.6203
0.6544	4.8571	17	0.5676	0.6709
0.6173	6.0	21	0.6543	0.5823
0.6173	6.8571	24	0.5310	0.7342
0.6173	8.0	28	0.6724	0.6076
0.5245	8.8571	31	0.6444	0.6582
0.5245	10.0	35	0.5027	0.7342
0.5245	10.8571	38	0.6328	0.6582
0.4554	12.0	42	0.4883	0.7595
0.4554	12.8571	45	0.6736	0.6582
0.4554	14.0	49	0.4584	0.7342
0.4575	14.8571	52	0.8099	0.6456
0.4575	16.0	56	0.4767	0.7468
0.4575	16.8571	59	0.6059	0.6835
0.3798	18.0	63	0.4863	0.7595
0.3798	18.8571	66	0.5636	0.7468
0.3419	20.0	70	0.4677	0.7342
0.3419	20.8571	73	0.4883	0.7089
0.3419	22.0	77	0.5549	0.7215
0.3079	22.8571	80	0.4324	0.7848
0.3079	24.0	84	0.6184	0.6709
0.3079	24.8571	87	0.6149	0.7089
0.2616	26.0	91	0.4488	0.7848
0.2616	26.8571	94	0.4368	0.7722
0.2616	28.0	98	0.4566	0.7722
0.2157	28.8571	101	0.4657	0.7848
0.2157	30.0	105	0.4514	0.7722
0.2157	30.8571	108	0.5083	0.7848
0.2258	32.0	112	0.5261	0.7848
0.2258	32.8571	115	0.5567	0.7595
0.2258	34.0	119	0.5566	0.8101
0.1972	34.8571	122	0.5495	0.8101
0.1972	36.0	126	0.4992	0.7975
0.1972	36.8571	129	0.5661	0.7595
0.1709	38.0	133	0.7326	0.7342
0.1709	38.8571	136	0.5635	0.8101
0.1537	40.0	140	0.8130	0.7468
0.1537	40.8571	143	0.6984	0.7848
0.1537	42.0	147	0.7777	0.7595
0.1687	42.8571	150	0.6452	0.8228
0.1687	44.0	154	0.8527	0.7215
0.1687	44.8571	157	0.6483	0.7975
0.1588	46.0	161	0.8185	0.7342
0.1588	46.8571	164	0.6821	0.7722
0.1588	48.0	168	0.7594	0.7342
0.144	48.8571	171	1.0232	0.7595
0.144	50.0	175	0.6178	0.7848
0.144	50.8571	178	0.6243	0.7595
0.1449	52.0	182	0.8159	0.7342
0.1449	52.8571	185	0.6664	0.7722
0.1449	54.0	189	0.7070	0.7342
0.144	54.8571	192	0.7361	0.7468
0.144	56.0	196	0.6656	0.7595
0.144	56.8571	199	0.7487	0.7468
0.1199	58.0	203	0.7993	0.7342
0.1199	58.8571	206	0.7426	0.7722
0.1258	60.0	210	0.7531	0.7975
0.1258	60.8571	213	0.7388	0.7848
0.1258	62.0	217	0.7395	0.7975
0.1392	62.8571	220	0.8238	0.7468
0.1392	64.0	224	0.9302	0.7215
0.1392	64.8571	227	0.7539	0.7722
0.1303	66.0	231	0.6739	0.8101
0.1303	66.8571	234	0.6627	0.7848
0.1303	68.0	238	0.6403	0.7848
0.1423	68.8571	241	0.6379	0.7975
0.1423	70.0	245	0.7658	0.7595
0.1423	70.8571	248	0.9195	0.7342
0.1019	72.0	252	0.7287	0.7722
0.1019	72.8571	255	0.6548	0.7975
0.1019	74.0	259	0.6534	0.7848
0.1286	74.8571	262	0.7331	0.7848
0.1286	76.0	266	0.7845	0.7595
0.1286	76.8571	269	0.7188	0.7975
0.1054	78.0	273	0.6595	0.7722
0.1054	78.8571	276	0.6623	0.7722
0.1053	80.0	280	0.7337	0.7722
0.1053	80.8571	283	0.8085	0.7468
0.1053	82.0	287	0.8201	0.7468
0.1086	82.8571	290	0.7947	0.7468
0.1086	84.0	294	0.7669	0.7722
0.1086	84.8571	297	0.7582	0.7595
0.1186	85.7143	300	0.7541	0.7595

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1