deit-base-distilled-patch16-224-85-fold5

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3071
• Accuracy: 0.9091

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	1.1685	0.3182
No log	2.0	4	0.7295	0.4545
No log	3.0	6	0.6641	0.7045
No log	4.0	8	0.7703	0.7045
0.7885	5.0	10	0.6750	0.7045
0.7885	6.0	12	0.6446	0.7045
0.7885	7.0	14	0.6919	0.7045
0.7885	8.0	16	0.6489	0.7045
0.7885	9.0	18	0.5245	0.7273
0.4488	10.0	20	0.8494	0.7045
0.4488	11.0	22	0.9086	0.6818
0.4488	12.0	24	0.5250	0.75
0.4488	13.0	26	0.5179	0.7727
0.4488	14.0	28	0.4423	0.7727
0.3387	15.0	30	0.5114	0.7273
0.3387	16.0	32	0.5048	0.75
0.3387	17.0	34	0.4997	0.7045
0.3387	18.0	36	0.4776	0.7045
0.3387	19.0	38	0.4138	0.7955
0.24	20.0	40	0.3220	0.8864
0.24	21.0	42	0.3363	0.8409
0.24	22.0	44	0.2958	0.8636
0.24	23.0	46	0.3098	0.8636
0.24	24.0	48	0.4030	0.8636
0.1524	25.0	50	0.3094	0.8636
0.1524	26.0	52	0.2721	0.8864
0.1524	27.0	54	0.3363	0.8636
0.1524	28.0	56	0.2731	0.8636
0.1524	29.0	58	0.5660	0.7955
0.1646	30.0	60	0.4949	0.8409
0.1646	31.0	62	0.4087	0.7727
0.1646	32.0	64	0.4467	0.8409
0.1646	33.0	66	0.4130	0.8182
0.1646	34.0	68	0.3727	0.8409
0.136	35.0	70	0.5894	0.7727
0.136	36.0	72	0.9462	0.75
0.136	37.0	74	0.5926	0.7273
0.136	38.0	76	0.3138	0.8864
0.136	39.0	78	0.4173	0.8864
0.163	40.0	80	0.3154	0.8636
0.163	41.0	82	0.3235	0.8636
0.163	42.0	84	0.3902	0.8182
0.163	43.0	86	0.3699	0.7955
0.163	44.0	88	0.4311	0.8182
0.1018	45.0	90	0.3071	0.9091
0.1018	46.0	92	0.2849	0.9091
0.1018	47.0	94	0.3226	0.8409
0.1018	48.0	96	0.2967	0.8409
0.1018	49.0	98	0.2936	0.8636
0.0957	50.0	100	0.2737	0.8864
0.0957	51.0	102	0.2845	0.8864
0.0957	52.0	104	0.3300	0.8409
0.0957	53.0	106	0.4029	0.8409
0.0957	54.0	108	0.4279	0.8182
0.1036	55.0	110	0.3900	0.8182
0.1036	56.0	112	0.4038	0.8636
0.1036	57.0	114	0.3569	0.8409
0.1036	58.0	116	0.5611	0.8182
0.1036	59.0	118	0.6900	0.8182
0.1048	60.0	120	0.5679	0.8182
0.1048	61.0	122	0.4567	0.8182
0.1048	62.0	124	0.3815	0.7955
0.1048	63.0	126	0.3546	0.7955
0.1048	64.0	128	0.3654	0.7955
0.0928	65.0	130	0.3337	0.8864
0.0928	66.0	132	0.4161	0.8409
0.0928	67.0	134	0.3615	0.8409
0.0928	68.0	136	0.4061	0.8182
0.0928	69.0	138	0.4191	0.8182
0.1091	70.0	140	0.3978	0.7955
0.1091	71.0	142	0.5168	0.75
0.1091	72.0	144	0.5268	0.75
0.1091	73.0	146	0.5667	0.7955
0.1091	74.0	148	0.5396	0.7727
0.1009	75.0	150	0.4807	0.75
0.1009	76.0	152	0.3957	0.8182
0.1009	77.0	154	0.3519	0.8636
0.1009	78.0	156	0.3654	0.8636
0.1009	79.0	158	0.3577	0.8409
0.0836	80.0	160	0.3216	0.8636
0.0836	81.0	162	0.3132	0.8409
0.0836	82.0	164	0.3003	0.8636
0.0836	83.0	166	0.3024	0.8636
0.0836	84.0	168	0.3214	0.8409
0.0928	85.0	170	0.3306	0.8182
0.0928	86.0	172	0.3284	0.8409
0.0928	87.0	174	0.3272	0.8182
0.0928	88.0	176	0.3261	0.8182
0.0928	89.0	178	0.3099	0.8409
0.0915	90.0	180	0.2928	0.8409
0.0915	91.0	182	0.2848	0.8409
0.0915	92.0	184	0.2827	0.8409
0.0915	93.0	186	0.2885	0.8636
0.0915	94.0	188	0.3084	0.8864
0.0775	95.0	190	0.3321	0.8409
0.0775	96.0	192	0.3358	0.8636
0.0775	97.0	194	0.3221	0.8409
0.0775	98.0	196	0.3096	0.8636
0.0775	99.0	198	0.3007	0.8864
0.091	100.0	200	0.2976	0.8864

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1