vit-base-patch16-224-ve-U12-b-80

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.8139
• Accuracy: 0.8478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.3850	0.3478
1.3848	2.0	13	1.3701	0.4783
1.3848	2.92	19	1.3196	0.5
1.3508	4.0	26	1.2287	0.4130
1.2282	4.92	32	1.1280	0.3913
1.2282	6.0	39	1.0625	0.3913
1.0677	6.92	45	0.9840	0.5
0.9278	8.0	52	0.8970	0.6957
0.9278	8.92	58	0.8530	0.7391
0.8003	10.0	65	0.7872	0.8043
0.6486	10.92	71	0.6974	0.8043
0.6486	12.0	78	0.6409	0.8043
0.514	12.92	84	0.6050	0.8261
0.3945	14.0	91	0.6589	0.7609
0.3945	14.92	97	0.6343	0.7609
0.337	16.0	104	0.7340	0.7174
0.2779	16.92	110	0.5629	0.8261
0.2779	18.0	117	0.5934	0.8261
0.2374	18.92	123	0.7080	0.7609
0.2201	20.0	130	0.7100	0.7391
0.2201	20.92	136	0.7673	0.7609
0.1889	22.0	143	0.7889	0.7391
0.1889	22.92	149	0.7971	0.7391
0.1463	24.0	156	0.6888	0.7826
0.1261	24.92	162	0.8399	0.7609
0.1261	26.0	169	0.7244	0.7826
0.1489	26.92	175	0.8311	0.7391
0.1132	28.0	182	0.7987	0.7609
0.1132	28.92	188	0.7380	0.8043
0.1279	30.0	195	0.8103	0.8043
0.0925	30.92	201	0.8462	0.7609
0.0925	32.0	208	0.8233	0.8043
0.0893	32.92	214	0.8241	0.7826
0.083	34.0	221	0.8443	0.7826
0.083	34.92	227	0.8429	0.7826
0.1044	36.0	234	0.9362	0.7609
0.0739	36.92	240	1.1173	0.7391
0.0739	38.0	247	0.7812	0.8261
0.0962	38.92	253	0.7595	0.8043
0.0869	40.0	260	0.8031	0.8261
0.0869	40.92	266	0.8359	0.8261
0.0837	42.0	273	0.8151	0.8261
0.0837	42.92	279	0.8295	0.8261
0.0535	44.0	286	0.8096	0.8261
0.0694	44.92	292	0.8352	0.8261
0.0694	46.0	299	0.8216	0.8261
0.0736	46.92	305	0.8683	0.8043
0.0705	48.0	312	0.8554	0.8261
0.0705	48.92	318	0.8139	0.8478
0.0559	50.0	325	0.9030	0.7826
0.0474	50.92	331	0.9053	0.7609
0.0474	52.0	338	0.8810	0.8261
0.0477	52.92	344	0.8912	0.8043
0.0529	54.0	351	0.9078	0.8043
0.0529	54.92	357	0.8804	0.8043
0.038	56.0	364	0.9498	0.7826
0.0407	56.92	370	0.9134	0.8043
0.0407	58.0	377	0.8452	0.8478
0.0353	58.92	383	0.8735	0.8261
0.0349	60.0	390	0.9153	0.8043
0.0349	60.92	396	0.9209	0.8043
0.0322	62.0	403	0.9091	0.8261
0.0322	62.92	409	0.9137	0.8261
0.0392	64.0	416	0.8896	0.8261
0.0419	64.92	422	0.8613	0.8478
0.0419	66.0	429	0.8844	0.8261
0.0518	66.92	435	0.9093	0.8043
0.0349	68.0	442	0.9082	0.8043
0.0349	68.92	448	0.8879	0.8261
0.0359	70.0	455	0.8809	0.8261
0.0377	70.92	461	0.8777	0.8261
0.0377	72.0	468	0.8845	0.8261
0.0324	72.92	474	0.8845	0.8261
0.0365	73.85	480	0.8850	0.8261

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0