deit-base-distilled-patch16-224-85-fold4

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.2141
• Accuracy: 0.9318

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.6100	0.7273
No log	2.0	4	0.6938	0.7045
No log	3.0	6	0.7568	0.7045
No log	4.0	8	0.6140	0.7045
0.5388	5.0	10	0.4976	0.75
0.5388	6.0	12	0.4809	0.7273
0.5388	7.0	14	0.5276	0.7273
0.5388	8.0	16	0.4455	0.7955
0.5388	9.0	18	0.3915	0.8409
0.4154	10.0	20	0.5070	0.7955
0.4154	11.0	22	0.3747	0.8182
0.4154	12.0	24	0.3027	0.8864
0.4154	13.0	26	0.3053	0.8636
0.4154	14.0	28	0.3194	0.8409
0.3258	15.0	30	0.3134	0.8864
0.3258	16.0	32	0.2925	0.8864
0.3258	17.0	34	0.2449	0.8864
0.3258	18.0	36	0.2308	0.8864
0.3258	19.0	38	0.2141	0.9318
0.2528	20.0	40	0.2330	0.9318
0.2528	21.0	42	0.2173	0.9318
0.2528	22.0	44	0.2450	0.9091
0.2528	23.0	46	0.2549	0.9091
0.2528	24.0	48	0.4341	0.75
0.175	25.0	50	0.2358	0.9091
0.175	26.0	52	0.2828	0.8864
0.175	27.0	54	0.2236	0.9091
0.175	28.0	56	0.2591	0.8636
0.175	29.0	58	0.2702	0.8864
0.169	30.0	60	0.2910	0.8636
0.169	31.0	62	0.3594	0.9091
0.169	32.0	64	0.4246	0.8864
0.169	33.0	66	0.2655	0.8864
0.169	34.0	68	0.2581	0.8864
0.1336	35.0	70	0.2494	0.8409
0.1336	36.0	72	0.2438	0.8636
0.1336	37.0	74	0.3246	0.8636
0.1336	38.0	76	0.2887	0.8409
0.1336	39.0	78	0.3559	0.8409
0.1281	40.0	80	0.3274	0.8864
0.1281	41.0	82	0.3371	0.8409
0.1281	42.0	84	0.3902	0.8409
0.1281	43.0	86	0.3100	0.8409
0.1281	44.0	88	0.3113	0.8636
0.136	45.0	90	0.3244	0.8409
0.136	46.0	92	0.3765	0.8864
0.136	47.0	94	0.3838	0.8864
0.136	48.0	96	0.3845	0.7955
0.136	49.0	98	0.3910	0.7955
0.0934	50.0	100	0.4889	0.8636
0.0934	51.0	102	0.6680	0.8182
0.0934	52.0	104	0.4264	0.8864
0.0934	53.0	106	0.3266	0.8182
0.0934	54.0	108	0.3168	0.8864
0.0999	55.0	110	0.3671	0.8182
0.0999	56.0	112	0.4684	0.8182
0.0999	57.0	114	0.4254	0.8182
0.0999	58.0	116	0.3195	0.8182
0.0999	59.0	118	0.3860	0.8864
0.1145	60.0	120	0.4805	0.8636
0.1145	61.0	122	0.3864	0.8182
0.1145	62.0	124	0.3347	0.8182
0.1145	63.0	126	0.3144	0.8182
0.1145	64.0	128	0.3267	0.8636
0.0769	65.0	130	0.3592	0.8636
0.0769	66.0	132	0.3520	0.8636
0.0769	67.0	134	0.3632	0.8636
0.0769	68.0	136	0.3955	0.8636
0.0769	69.0	138	0.4053	0.8182
0.0976	70.0	140	0.4272	0.8636
0.0976	71.0	142	0.4345	0.8409
0.0976	72.0	144	0.3943	0.8636
0.0976	73.0	146	0.3827	0.8636
0.0976	74.0	148	0.4133	0.8409
0.0981	75.0	150	0.4311	0.8409
0.0981	76.0	152	0.4126	0.8409
0.0981	77.0	154	0.3651	0.8636
0.0981	78.0	156	0.3511	0.8182
0.0981	79.0	158	0.3625	0.8636
0.085	80.0	160	0.3607	0.8636
0.085	81.0	162	0.3470	0.8409
0.085	82.0	164	0.3639	0.8409
0.085	83.0	166	0.3750	0.8409
0.085	84.0	168	0.3726	0.7955
0.0831	85.0	170	0.3740	0.8182
0.0831	86.0	172	0.3807	0.8636
0.0831	87.0	174	0.3875	0.8636
0.0831	88.0	176	0.3886	0.8409
0.0831	89.0	178	0.4017	0.7955
0.0811	90.0	180	0.4271	0.7955
0.0811	91.0	182	0.4293	0.7955
0.0811	92.0	184	0.4243	0.7727
0.0811	93.0	186	0.4088	0.7727
0.0811	94.0	188	0.3986	0.7955
0.0692	95.0	190	0.3963	0.8182
0.0692	96.0	192	0.3987	0.8636
0.0692	97.0	194	0.4020	0.8636
0.0692	98.0	196	0.4015	0.8636
0.0692	99.0	198	0.4009	0.8636
0.0644	100.0	200	0.4002	0.8636

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1