deit-base-distilled-patch16-224-75-fold1

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3098
• Accuracy: 0.9070

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.6477	0.6512
No log	2.0	4	0.6528	0.6977
No log	3.0	6	0.8096	0.6977
No log	4.0	8	0.7679	0.6977
0.5994	5.0	10	0.5482	0.6977
0.5994	6.0	12	0.4984	0.7442
0.5994	7.0	14	0.6156	0.6977
0.5994	8.0	16	0.5307	0.7674
0.5994	9.0	18	0.4036	0.7674
0.3806	10.0	20	0.4241	0.7907
0.3806	11.0	22	0.4263	0.8140
0.3806	12.0	24	0.6778	0.7442
0.3806	13.0	26	0.5885	0.7674
0.3806	14.0	28	0.6048	0.7907
0.273	15.0	30	0.5110	0.8140
0.273	16.0	32	0.3793	0.7442
0.273	17.0	34	0.3635	0.7907
0.273	18.0	36	0.3863	0.8140
0.273	19.0	38	0.3788	0.8372
0.2388	20.0	40	0.3390	0.8140
0.2388	21.0	42	0.4593	0.7907
0.2388	22.0	44	0.3441	0.8605
0.2388	23.0	46	0.5483	0.7907
0.2388	24.0	48	0.6399	0.7907
0.189	25.0	50	0.3333	0.8605
0.189	26.0	52	0.3326	0.8605
0.189	27.0	54	0.4150	0.7907
0.189	28.0	56	0.3420	0.8837
0.189	29.0	58	0.3649	0.8372
0.1718	30.0	60	0.3651	0.8605
0.1718	31.0	62	0.4676	0.8140
0.1718	32.0	64	0.3543	0.8605
0.1718	33.0	66	0.3209	0.8140
0.1718	34.0	68	0.3420	0.8605
0.1466	35.0	70	0.3800	0.8372
0.1466	36.0	72	0.6547	0.8140
0.1466	37.0	74	0.9743	0.7674
0.1466	38.0	76	0.6677	0.7907
0.1466	39.0	78	0.5691	0.8140
0.119	40.0	80	0.4796	0.8605
0.119	41.0	82	0.3243	0.8837
0.119	42.0	84	0.2969	0.8605
0.119	43.0	86	0.3637	0.8372
0.119	44.0	88	0.3098	0.9070
0.1123	45.0	90	0.3954	0.8837
0.1123	46.0	92	0.3197	0.9070
0.1123	47.0	94	0.3188	0.8372
0.1123	48.0	96	0.3100	0.8372
0.1123	49.0	98	0.3653	0.8605
0.1136	50.0	100	0.3527	0.8837
0.1136	51.0	102	0.3152	0.8605
0.1136	52.0	104	0.3277	0.8605
0.1136	53.0	106	0.3221	0.8837
0.1136	54.0	108	0.3438	0.8605
0.0858	55.0	110	0.4683	0.8605
0.0858	56.0	112	0.4511	0.8605
0.0858	57.0	114	0.3486	0.8605
0.0858	58.0	116	0.3594	0.8837
0.0858	59.0	118	0.3914	0.8605
0.084	60.0	120	0.4257	0.8837
0.084	61.0	122	0.4505	0.8837
0.084	62.0	124	0.4038	0.8605
0.084	63.0	126	0.3745	0.8372
0.084	64.0	128	0.3774	0.8140
0.0938	65.0	130	0.3712	0.8140
0.0938	66.0	132	0.3736	0.8140
0.0938	67.0	134	0.3840	0.8605
0.0938	68.0	136	0.3902	0.8605
0.0938	69.0	138	0.4105	0.8605
0.055	70.0	140	0.4498	0.8605
0.055	71.0	142	0.4954	0.8605
0.055	72.0	144	0.6397	0.8605
0.055	73.0	146	0.6271	0.8605
0.055	74.0	148	0.4821	0.8605
0.0755	75.0	150	0.3699	0.9070
0.0755	76.0	152	0.3303	0.8605
0.0755	77.0	154	0.3282	0.8837
0.0755	78.0	156	0.3181	0.8837
0.0755	79.0	158	0.3083	0.8605
0.0603	80.0	160	0.3170	0.8372
0.0603	81.0	162	0.3397	0.8372
0.0603	82.0	164	0.3538	0.8372
0.0603	83.0	166	0.3461	0.8372
0.0603	84.0	168	0.3337	0.8140
0.0653	85.0	170	0.3330	0.8372
0.0653	86.0	172	0.3451	0.8837
0.0653	87.0	174	0.3612	0.8837
0.0653	88.0	176	0.3822	0.8605
0.0653	89.0	178	0.3875	0.8605
0.0571	90.0	180	0.3845	0.8605
0.0571	91.0	182	0.3642	0.8837
0.0571	92.0	184	0.3529	0.8837
0.0571	93.0	186	0.3471	0.8837
0.0571	94.0	188	0.3540	0.8837
0.069	95.0	190	0.3609	0.8837
0.069	96.0	192	0.3609	0.8837
0.069	97.0	194	0.3634	0.8837
0.069	98.0	196	0.3627	0.8837
0.069	99.0	198	0.3609	0.8837
0.0667	100.0	200	0.3604	0.8837

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1