vit-base-patch16-224-finetuned-context-classifier

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.7157
• Accuracy: 0.8188

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 512
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.3586	2.0	10	1.2322	0.3916
1.0841	4.0	20	0.8444	0.6958
0.7282	6.0	30	0.5498	0.7767
0.4768	8.0	40	0.4273	0.8155
0.3332	10.0	50	0.4059	0.8220
0.242	12.0	60	0.4272	0.8252
0.1737	14.0	70	0.4372	0.8188
0.1266	16.0	80	0.4495	0.8123
0.1089	18.0	90	0.4877	0.8091
0.0837	20.0	100	0.5318	0.8058
0.0687	22.0	110	0.5300	0.7961
0.0667	24.0	120	0.6253	0.7994
0.0581	26.0	130	0.5495	0.8220
0.0574	28.0	140	0.5646	0.8188
0.0564	30.0	150	0.5990	0.8252
0.0492	32.0	160	0.6436	0.8155
0.0406	34.0	170	0.6225	0.8091
0.0411	36.0	180	0.6168	0.8123
0.0381	38.0	190	0.6731	0.8123
0.0358	40.0	200	0.6198	0.7961
0.0354	42.0	210	0.6216	0.8091
0.0358	44.0	220	0.6933	0.8091
0.037	46.0	230	0.6488	0.8188
0.0344	48.0	240	0.6546	0.8220
0.0335	50.0	250	0.6399	0.7994
0.0297	52.0	260	0.6553	0.8123
0.0318	54.0	270	0.6996	0.7896
0.0254	56.0	280	0.6809	0.7961
0.0322	58.0	290	0.7048	0.7896
0.024	60.0	300	0.6869	0.8123
0.0255	62.0	310	0.7099	0.8058
0.0266	64.0	320	0.6894	0.8091
0.0243	66.0	330	0.7604	0.8091
0.0232	68.0	340	0.6983	0.8123
0.019	70.0	350	0.6834	0.8091
0.0235	72.0	360	0.7102	0.8091
0.0262	74.0	370	0.6902	0.8155
0.0206	76.0	380	0.6662	0.8091
0.0238	78.0	390	0.7109	0.8220
0.0202	80.0	400	0.7061	0.8058
0.0204	82.0	410	0.7291	0.8155
0.0231	84.0	420	0.7103	0.8091
0.0217	86.0	430	0.7050	0.8123
0.021	88.0	440	0.7037	0.8155
0.0207	90.0	450	0.6996	0.8058
0.0163	92.0	460	0.7137	0.8091
0.0181	94.0	470	0.7153	0.8155
0.0225	96.0	480	0.7105	0.8123
0.0185	98.0	490	0.7140	0.8155
0.0219	100.0	500	0.7157	0.8188

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.4
• Tokenizers 0.14.1