beit-base-patch16-224-65-fold4

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5415
• Accuracy: 0.8732

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9231	3	0.7415	0.5352
No log	1.8462	6	0.7177	0.4507
No log	2.7692	9	0.6709	0.6056
0.748	4.0	13	0.6333	0.6338
0.748	4.9231	16	0.6162	0.7324
0.748	5.8462	19	0.6303	0.6338
0.6397	6.7692	22	0.5950	0.6761
0.6397	8.0	26	0.6325	0.6056
0.6397	8.9231	29	0.5799	0.7042
0.5957	9.8462	32	0.5793	0.6901
0.5957	10.7692	35	0.5869	0.7183
0.5957	12.0	39	0.6195	0.5775
0.5676	12.9231	42	0.5940	0.6479
0.5676	13.8462	45	0.6612	0.6197
0.5676	14.7692	48	0.5598	0.7465
0.5952	16.0	52	0.5472	0.7465
0.5952	16.9231	55	0.4823	0.7887
0.5952	17.8462	58	0.6493	0.6901
0.4908	18.7692	61	0.5539	0.7465
0.4908	20.0	65	0.5406	0.7606
0.4908	20.9231	68	0.5443	0.7606
0.4474	21.8462	71	0.6548	0.7042
0.4474	22.7692	74	0.4924	0.7746
0.4474	24.0	78	0.4671	0.8169
0.4106	24.9231	81	0.4117	0.8310
0.4106	25.8462	84	0.4630	0.8592
0.4106	26.7692	87	0.4915	0.8310
0.3163	28.0	91	0.6336	0.8028
0.3163	28.9231	94	0.5920	0.7887
0.3163	29.8462	97	0.5653	0.8028
0.3234	30.7692	100	0.6411	0.7746
0.3234	32.0	104	0.6728	0.7887
0.3234	32.9231	107	0.5503	0.8028
0.2969	33.8462	110	0.4914	0.8310
0.2969	34.7692	113	0.5952	0.8169
0.2969	36.0	117	0.7161	0.7746
0.2325	36.9231	120	0.6517	0.7746
0.2325	37.8462	123	0.5832	0.7887
0.2325	38.7692	126	0.6309	0.7746
0.2447	40.0	130	0.8011	0.7465
0.2447	40.9231	133	0.6085	0.7887
0.2447	41.8462	136	0.6470	0.7606
0.2447	42.7692	139	0.7744	0.7746
0.2217	44.0	143	0.5730	0.8310
0.2217	44.9231	146	0.5577	0.8169
0.2217	45.8462	149	0.5226	0.8451
0.2231	46.7692	152	0.5115	0.8310
0.2231	48.0	156	0.5415	0.8732
0.2231	48.9231	159	0.5971	0.8310
0.2014	49.8462	162	0.8717	0.7606
0.2014	50.7692	165	0.7063	0.7887
0.2014	52.0	169	0.6917	0.7887
0.1827	52.9231	172	0.6880	0.7887
0.1827	53.8462	175	0.7027	0.8028
0.1827	54.7692	178	0.6764	0.8310
0.1558	56.0	182	0.7398	0.7887
0.1558	56.9231	185	0.7787	0.8169
0.1558	57.8462	188	0.7678	0.8169
0.1637	58.7692	191	0.7898	0.7606
0.1637	60.0	195	0.7105	0.8310
0.1637	60.9231	198	0.7262	0.8592
0.1591	61.8462	201	0.7464	0.8169
0.1591	62.7692	204	0.7233	0.8310
0.1591	64.0	208	0.7263	0.8310
0.1521	64.9231	211	0.7377	0.8028
0.1521	65.8462	214	0.7267	0.8310
0.1521	66.7692	217	0.7178	0.8169
0.157	68.0	221	0.8585	0.7887
0.157	68.9231	224	0.8629	0.7887
0.157	69.8462	227	0.7329	0.8028
0.1593	70.7692	230	0.6997	0.8310
0.1593	72.0	234	0.8074	0.8028
0.1593	72.9231	237	1.0352	0.7887
0.134	73.8462	240	1.0472	0.7887
0.134	74.7692	243	0.7477	0.8169
0.134	76.0	247	0.7357	0.8310
0.1386	76.9231	250	0.8497	0.7887
0.1386	77.8462	253	0.9464	0.7746
0.1386	78.7692	256	0.8535	0.7887
0.1246	80.0	260	0.7998	0.8310
0.1246	80.9231	263	0.8214	0.8310
0.1246	81.8462	266	0.8374	0.8028
0.1246	82.7692	269	0.8597	0.8028
0.1271	84.0	273	0.8437	0.8028
0.1271	84.9231	276	0.8370	0.8028
0.1271	85.8462	279	0.8298	0.8028
0.1274	86.7692	282	0.8340	0.8028
0.1274	88.0	286	0.8462	0.8028
0.1274	88.9231	289	0.8594	0.8028
0.1251	89.8462	292	0.8504	0.8028
0.1251	90.7692	295	0.8480	0.8028
0.1251	92.0	299	0.8471	0.8028
0.1207	92.3077	300	0.8469	0.8028

Framework versions

• Transformers 4.40.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1