vit-base-patch16-224-RU2-40

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.2003
• Accuracy: 0.8333

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.3226	0.99	38	1.2293	0.6
0.9048	2.0	77	0.7969	0.7
0.4039	2.99	115	0.6800	0.7167
0.281	4.0	154	0.8892	0.7667
0.1755	4.99	192	0.9072	0.7333
0.1035	6.0	231	0.8036	0.8167
0.1275	6.99	269	0.8627	0.8
0.107	8.0	308	0.8713	0.8
0.0984	8.99	346	0.9660	0.8
0.0823	10.0	385	1.0704	0.7833
0.0771	10.99	423	0.9409	0.7667
0.0527	12.0	462	1.0052	0.7833
0.0708	12.99	500	0.9578	0.8
0.0562	14.0	539	1.0712	0.8167
0.0467	14.99	577	1.0586	0.8167
0.0445	16.0	616	1.2066	0.7667
0.0474	16.99	654	1.1863	0.75
0.0263	18.0	693	1.1207	0.8167
0.0307	18.99	731	1.1813	0.8167
0.0393	20.0	770	1.3761	0.75
0.0475	20.99	808	1.3008	0.7667
0.0215	22.0	847	1.2625	0.7333
0.0311	22.99	885	1.1508	0.8
0.027	24.0	924	1.3035	0.7667
0.0251	24.99	962	1.2270	0.7667
0.0161	26.0	1001	1.1470	0.8167
0.0258	26.99	1039	1.1473	0.8167
0.0142	28.0	1078	1.2326	0.7667
0.0151	28.99	1116	1.3978	0.7667
0.021	30.0	1155	1.2003	0.8333
0.0158	30.99	1193	1.2488	0.7667
0.0163	32.0	1232	1.3232	0.75
0.0143	32.99	1270	1.2467	0.8
0.02	34.0	1309	1.3176	0.7833
0.0128	34.99	1347	1.3083	0.7667
0.0144	36.0	1386	1.3080	0.7667
0.0109	36.99	1424	1.2999	0.8
0.0082	38.0	1463	1.2718	0.8
0.0064	38.99	1501	1.2588	0.7667
0.0097	39.48	1520	1.2597	0.7667

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0