vit-base-patch16-224-dmae-va-U3-40A

This model is a fine-tuned version of google/vit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0213
• Accuracy: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	7	1.3292	0.3737
1.3407	2.0	14	1.1079	0.5152
1.3407	3.0	21	0.8918	0.6263
0.9919	4.0	28	0.6447	0.7879
0.9919	5.0	35	0.4502	0.8283
0.5761	6.0	42	0.2720	0.9192
0.3111	7.0	49	0.2302	0.9293
0.3111	8.0	56	0.1650	0.9495
0.204	9.0	63	0.1503	0.9495
0.204	10.0	70	0.0814	0.9798
0.1518	11.0	77	0.0604	0.9798
0.1272	12.0	84	0.1265	0.9495
0.1272	13.0	91	0.0518	0.9798
0.1379	14.0	98	0.0448	0.9899
0.1379	15.0	105	0.0361	0.9899
0.092	16.0	112	0.0322	0.9899
0.092	17.0	119	0.0213	1.0
0.0762	18.0	126	0.0469	0.9899
0.0954	19.0	133	0.0615	0.9899
0.0954	20.0	140	0.0313	0.9899
0.0795	21.0	147	0.0381	0.9899
0.0795	22.0	154	0.0138	1.0
0.077	23.0	161	0.0170	1.0
0.0675	24.0	168	0.0107	1.0
0.0675	25.0	175	0.0193	0.9899
0.0659	26.0	182	0.0255	0.9899
0.0659	27.0	189	0.0201	0.9899
0.0758	28.0	196	0.0325	0.9899
0.0758	29.0	203	0.0110	1.0
0.0589	30.0	210	0.0159	1.0
0.0521	31.0	217	0.0319	0.9899
0.0521	32.0	224	0.0294	0.9798
0.0618	33.0	231	0.0392	0.9798
0.0618	34.0	238	0.0269	0.9899
0.0422	35.0	245	0.0210	0.9899
0.0551	36.0	252	0.0178	0.9899
0.0551	37.0	259	0.0159	0.9899
0.0518	38.0	266	0.0124	0.9899
0.0518	39.0	273	0.0112	1.0
0.0313	40.0	280	0.0110	1.0

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2