deit-base-distilled-patch16-224-75-fold5

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.2788
• Accuracy: 0.9535

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.8041	0.3488
No log	2.0	4	0.6042	0.7209
No log	3.0	6	0.6582	0.6977
No log	4.0	8	0.7007	0.6977
0.6827	5.0	10	0.5653	0.6977
0.6827	6.0	12	0.4345	0.8140
0.6827	7.0	14	0.4493	0.8140
0.6827	8.0	16	0.4764	0.8140
0.6827	9.0	18	0.3636	0.8372
0.4147	10.0	20	0.2769	0.8605
0.4147	11.0	22	0.3538	0.8372
0.4147	12.0	24	0.3320	0.8605
0.4147	13.0	26	0.2668	0.8372
0.4147	14.0	28	0.2823	0.8140
0.3405	15.0	30	0.4864	0.8372
0.3405	16.0	32	0.4416	0.8372
0.3405	17.0	34	0.3585	0.8140
0.3405	18.0	36	0.4769	0.8372
0.3405	19.0	38	0.5918	0.8372
0.3074	20.0	40	0.4116	0.8605
0.3074	21.0	42	0.4578	0.8372
0.3074	22.0	44	0.4294	0.8605
0.3074	23.0	46	0.5401	0.8372
0.3074	24.0	48	0.2069	0.8837
0.2155	25.0	50	0.2216	0.8837
0.2155	26.0	52	0.3837	0.8837
0.2155	27.0	54	0.2473	0.9070
0.2155	28.0	56	0.2233	0.9070
0.2155	29.0	58	0.3489	0.8837
0.1874	30.0	60	0.4635	0.8605
0.1874	31.0	62	0.2491	0.8837
0.1874	32.0	64	0.2564	0.8837
0.1874	33.0	66	0.3655	0.8837
0.1874	34.0	68	0.2498	0.9302
0.1324	35.0	70	0.2922	0.8837
0.1324	36.0	72	0.5368	0.8837
0.1324	37.0	74	0.5739	0.8837
0.1324	38.0	76	0.5049	0.8837
0.1324	39.0	78	0.5903	0.8837
0.1222	40.0	80	0.4886	0.8837
0.1222	41.0	82	0.4174	0.8837
0.1222	42.0	84	0.5429	0.8605
0.1222	43.0	86	0.6897	0.8605
0.1222	44.0	88	0.6805	0.8605
0.1008	45.0	90	0.4073	0.8837
0.1008	46.0	92	0.4161	0.8837
0.1008	47.0	94	0.6485	0.8837
0.1008	48.0	96	0.6746	0.8837
0.1008	49.0	98	0.4433	0.8837
0.117	50.0	100	0.2788	0.9535
0.117	51.0	102	0.3441	0.8837
0.117	52.0	104	0.4663	0.8605
0.117	53.0	106	0.3300	0.9070
0.117	54.0	108	0.2531	0.9535
0.0879	55.0	110	0.2261	0.9535
0.0879	56.0	112	0.3490	0.9070
0.0879	57.0	114	0.5060	0.8837
0.0879	58.0	116	0.4847	0.8837
0.0879	59.0	118	0.3604	0.8837
0.0898	60.0	120	0.3626	0.8837
0.0898	61.0	122	0.4641	0.8837
0.0898	62.0	124	0.4606	0.8837
0.0898	63.0	126	0.3419	0.8837
0.0898	64.0	128	0.3331	0.9070
0.095	65.0	130	0.3804	0.8837
0.095	66.0	132	0.3783	0.8837
0.095	67.0	134	0.3524	0.8837
0.095	68.0	136	0.4232	0.8605
0.095	69.0	138	0.4157	0.8605
0.0735	70.0	140	0.4937	0.8605
0.0735	71.0	142	0.4616	0.8605
0.0735	72.0	144	0.4388	0.8837
0.0735	73.0	146	0.3811	0.8837
0.0735	74.0	148	0.2522	0.9302
0.0855	75.0	150	0.1677	0.9535
0.0855	76.0	152	0.1653	0.9535
0.0855	77.0	154	0.2173	0.9535
0.0855	78.0	156	0.3596	0.8837
0.0855	79.0	158	0.4653	0.8605
0.077	80.0	160	0.4739	0.8837
0.077	81.0	162	0.3609	0.8837
0.077	82.0	164	0.2334	0.9535
0.077	83.0	166	0.2125	0.9535
0.077	84.0	168	0.2595	0.9535
0.0731	85.0	170	0.3702	0.8837
0.0731	86.0	172	0.4634	0.8837
0.0731	87.0	174	0.5254	0.8837
0.0731	88.0	176	0.5813	0.8837
0.0731	89.0	178	0.5628	0.8837
0.078	90.0	180	0.5479	0.8837
0.078	91.0	182	0.5216	0.8837
0.078	92.0	184	0.5098	0.8837
0.078	93.0	186	0.5244	0.8837
0.078	94.0	188	0.5472	0.8837
0.079	95.0	190	0.5590	0.8837
0.079	96.0	192	0.5589	0.8837
0.079	97.0	194	0.5455	0.8837
0.079	98.0	196	0.5322	0.8837
0.079	99.0	198	0.5215	0.8837
0.0477	100.0	200	0.5202	0.8837

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1