deit-base-distilled-patch16-224-85-fold2

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.1991
• Accuracy: 0.9318

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	1.2745	0.2727
No log	2.0	4	0.8028	0.4091
No log	3.0	6	0.7456	0.7045
No log	4.0	8	0.7982	0.7045
0.7325	5.0	10	0.6233	0.7045
0.7325	6.0	12	0.5093	0.7273
0.7325	7.0	14	0.5566	0.7045
0.7325	8.0	16	0.6839	0.7045
0.7325	9.0	18	0.4821	0.75
0.4472	10.0	20	0.4365	0.7727
0.4472	11.0	22	0.5158	0.7273
0.4472	12.0	24	0.4196	0.8182
0.4472	13.0	26	0.3599	0.8409
0.4472	14.0	28	0.3604	0.8636
0.3483	15.0	30	0.3634	0.8182
0.3483	16.0	32	0.2803	0.8864
0.3483	17.0	34	0.2592	0.8864
0.3483	18.0	36	0.2655	0.9091
0.3483	19.0	38	0.2333	0.8864
0.2514	20.0	40	0.2590	0.8636
0.2514	21.0	42	0.2642	0.8864
0.2514	22.0	44	0.2637	0.8864
0.2514	23.0	46	0.1991	0.9318
0.2514	24.0	48	0.1941	0.9091
0.1847	25.0	50	0.1868	0.8864
0.1847	26.0	52	0.1828	0.8864
0.1847	27.0	54	0.1711	0.8864
0.1847	28.0	56	0.2423	0.8864
0.1847	29.0	58	0.2162	0.8864
0.1501	30.0	60	0.1854	0.9091
0.1501	31.0	62	0.3071	0.8636
0.1501	32.0	64	0.2435	0.8864
0.1501	33.0	66	0.1728	0.9091
0.1501	34.0	68	0.1644	0.9091
0.13	35.0	70	0.2768	0.8409
0.13	36.0	72	0.1539	0.9318
0.13	37.0	74	0.2580	0.9091
0.13	38.0	76	0.1783	0.8864
0.13	39.0	78	0.1782	0.8636
0.1357	40.0	80	0.2035	0.8864
0.1357	41.0	82	0.2117	0.8864
0.1357	42.0	84	0.1793	0.9091
0.1357	43.0	86	0.2002	0.9091
0.1357	44.0	88	0.2366	0.8864
0.105	45.0	90	0.2008	0.9318
0.105	46.0	92	0.2368	0.8864
0.105	47.0	94	0.2142	0.8864
0.105	48.0	96	0.2117	0.8864
0.105	49.0	98	0.2621	0.8864
0.1091	50.0	100	0.2231	0.8864
0.1091	51.0	102	0.1946	0.9318
0.1091	52.0	104	0.2001	0.9318
0.1091	53.0	106	0.2031	0.9091
0.1091	54.0	108	0.2078	0.9091
0.1054	55.0	110	0.2250	0.9091
0.1054	56.0	112	0.2180	0.9091
0.1054	57.0	114	0.1915	0.9318
0.1054	58.0	116	0.2227	0.8864
0.1054	59.0	118	0.2352	0.8864
0.0982	60.0	120	0.2329	0.8864
0.0982	61.0	122	0.2135	0.9091
0.0982	62.0	124	0.1949	0.8864
0.0982	63.0	126	0.2149	0.9318
0.0982	64.0	128	0.2435	0.9091
0.0808	65.0	130	0.2541	0.9091
0.0808	66.0	132	0.2447	0.9091
0.0808	67.0	134	0.1904	0.9318
0.0808	68.0	136	0.2437	0.9091
0.0808	69.0	138	0.3593	0.8864
0.0843	70.0	140	0.4187	0.8864
0.0843	71.0	142	0.3510	0.8864
0.0843	72.0	144	0.2315	0.9091
0.0843	73.0	146	0.2049	0.9091
0.0843	74.0	148	0.2150	0.9091
0.0942	75.0	150	0.2116	0.9091
0.0942	76.0	152	0.2014	0.9091
0.0942	77.0	154	0.2198	0.9091
0.0942	78.0	156	0.2538	0.9318
0.0942	79.0	158	0.2755	0.9318
0.0884	80.0	160	0.2491	0.9091
0.0884	81.0	162	0.2100	0.9091
0.0884	82.0	164	0.1977	0.9091
0.0884	83.0	166	0.1979	0.9091
0.0884	84.0	168	0.2145	0.9091
0.0637	85.0	170	0.2192	0.9091
0.0637	86.0	172	0.2055	0.9318
0.0637	87.0	174	0.1994	0.9318
0.0637	88.0	176	0.1975	0.9091
0.0637	89.0	178	0.1974	0.9091
0.0923	90.0	180	0.1965	0.9091
0.0923	91.0	182	0.1925	0.9091
0.0923	92.0	184	0.1942	0.9091
0.0923	93.0	186	0.1969	0.9318
0.0923	94.0	188	0.1949	0.9318
0.0657	95.0	190	0.1904	0.9318
0.0657	96.0	192	0.1877	0.9091
0.0657	97.0	194	0.1885	0.9091
0.0657	98.0	196	0.1902	0.9318
0.0657	99.0	198	0.1922	0.9318
0.0822	100.0	200	0.1932	0.9318

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1