deit-base-distilled-patch16-224-85-fold1

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.2307
• Accuracy: 0.9545

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	1.1776	0.25
No log	2.0	4	0.8165	0.3182
No log	3.0	6	0.7525	0.7045
No log	4.0	8	0.8074	0.7045
0.7495	5.0	10	0.6842	0.7045
0.7495	6.0	12	0.5606	0.6818
0.7495	7.0	14	0.5377	0.6818
0.7495	8.0	16	0.5761	0.7045
0.7495	9.0	18	0.5866	0.7045
0.4611	10.0	20	0.4894	0.7273
0.4611	11.0	22	0.6216	0.7273
0.4611	12.0	24	0.6272	0.7273
0.4611	13.0	26	0.4476	0.7727
0.4611	14.0	28	0.4451	0.75
0.3766	15.0	30	0.4370	0.7727
0.3766	16.0	32	0.4937	0.75
0.3766	17.0	34	0.4977	0.7955
0.3766	18.0	36	0.4551	0.8409
0.3766	19.0	38	0.3776	0.7727
0.3147	20.0	40	0.3347	0.8409
0.3147	21.0	42	0.5178	0.7727
0.3147	22.0	44	0.3430	0.8409
0.3147	23.0	46	0.3091	0.8409
0.3147	24.0	48	0.3995	0.8864
0.2176	25.0	50	0.2971	0.8409
0.2176	26.0	52	0.3137	0.8864
0.2176	27.0	54	0.2694	0.8864
0.2176	28.0	56	0.2589	0.8864
0.2176	29.0	58	0.3612	0.8636
0.1855	30.0	60	0.3406	0.8636
0.1855	31.0	62	0.4738	0.8864
0.1855	32.0	64	0.7612	0.7955
0.1855	33.0	66	0.5307	0.8864
0.1855	34.0	68	0.3346	0.8636
0.2006	35.0	70	0.3562	0.8409
0.2006	36.0	72	0.5255	0.8409
0.2006	37.0	74	0.3795	0.8409
0.2006	38.0	76	0.2924	0.9091
0.2006	39.0	78	0.2921	0.8864
0.161	40.0	80	0.3895	0.8409
0.161	41.0	82	0.3421	0.8182
0.161	42.0	84	0.2674	0.8864
0.161	43.0	86	0.2586	0.8864
0.161	44.0	88	0.4520	0.8409
0.1588	45.0	90	0.4300	0.8409
0.1588	46.0	92	0.2424	0.9318
0.1588	47.0	94	0.2645	0.9318
0.1588	48.0	96	0.2531	0.8864
0.1588	49.0	98	0.2614	0.8864
0.1103	50.0	100	0.3024	0.8864
0.1103	51.0	102	0.2797	0.9091
0.1103	52.0	104	0.2307	0.9545
0.1103	53.0	106	0.2635	0.9091
0.1103	54.0	108	0.5111	0.8409
0.1201	55.0	110	0.5371	0.8409
0.1201	56.0	112	0.2940	0.8864
0.1201	57.0	114	0.3015	0.9091
0.1201	58.0	116	0.2631	0.8864
0.1201	59.0	118	0.2830	0.8864
0.1037	60.0	120	0.3202	0.8636
0.1037	61.0	122	0.3526	0.8636
0.1037	62.0	124	0.3975	0.8409
0.1037	63.0	126	0.4785	0.8409
0.1037	64.0	128	0.4306	0.8636
0.1	65.0	130	0.3230	0.8636
0.1	66.0	132	0.3007	0.8864
0.1	67.0	134	0.2669	0.8864
0.1	68.0	136	0.2335	0.8864
0.1	69.0	138	0.1845	0.8864
0.0984	70.0	140	0.2261	0.8864
0.0984	71.0	142	0.3015	0.8864
0.0984	72.0	144	0.3138	0.8864
0.0984	73.0	146	0.2444	0.8864
0.0984	74.0	148	0.2060	0.9091
0.0826	75.0	150	0.2024	0.9318
0.0826	76.0	152	0.2503	0.8864
0.0826	77.0	154	0.2499	0.8864
0.0826	78.0	156	0.2099	0.9091
0.0826	79.0	158	0.2240	0.9091
0.0701	80.0	160	0.2228	0.9091
0.0701	81.0	162	0.2337	0.9091
0.0701	82.0	164	0.2113	0.9318
0.0701	83.0	166	0.1977	0.9091
0.0701	84.0	168	0.2021	0.9091
0.0846	85.0	170	0.2330	0.9318
0.0846	86.0	172	0.2333	0.9318
0.0846	87.0	174	0.2130	0.9318
0.0846	88.0	176	0.2090	0.9091
0.0846	89.0	178	0.2114	0.9091
0.0932	90.0	180	0.2061	0.9091
0.0932	91.0	182	0.2174	0.9091
0.0932	92.0	184	0.2429	0.9091
0.0932	93.0	186	0.2459	0.9091
0.0932	94.0	188	0.2293	0.9318
0.0742	95.0	190	0.2127	0.9318
0.0742	96.0	192	0.2014	0.9091
0.0742	97.0	194	0.2015	0.9091
0.0742	98.0	196	0.2063	0.9318
0.0742	99.0	198	0.2088	0.9318
0.0701	100.0	200	0.2096	0.9318

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1