vit-base-patch16-224-dmae-va-U5-42D

This model is a fine-tuned version of google/vit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0383
• Accuracy: 0.55

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.0970	0.5167
1.3527	1.94	15	1.0383	0.55
1.3527	2.97	23	1.2351	0.4167
1.3013	4.0	31	1.3025	0.3333
1.3706	4.9	38	1.3800	0.2167
1.3706	5.94	46	1.4609	0.1833
1.4415	6.97	54	1.3718	0.4333
1.3602	8.0	62	1.3173	0.3167
1.3602	8.9	69	1.2827	0.4
1.3079	9.94	77	1.3167	0.3167
1.3247	10.97	85	1.2579	0.4
1.3247	12.0	93	1.3202	0.2
1.3102	12.9	100	1.2354	0.45
1.2807	13.94	108	1.3610	0.25
1.2807	14.97	116	1.2803	0.4
1.2774	16.0	124	1.3338	0.2167
1.2774	16.9	131	1.2549	0.35
1.2596	17.94	139	1.2693	0.3667
1.2413	18.97	147	1.3005	0.2167
1.2413	20.0	155	1.2299	0.4333
1.262	20.9	162	1.3454	0.2667
1.2261	21.94	170	1.2818	0.3167
1.2261	22.97	178	1.2498	0.4333
1.2405	24.0	186	1.3376	0.3167
1.2245	24.9	193	1.2595	0.3667
1.2245	25.94	201	1.3319	0.4
1.2034	26.97	209	1.2528	0.3833
1.1818	28.0	217	1.3656	0.3667
1.1818	28.9	224	1.2501	0.3833
1.1479	29.94	232	1.3241	0.3
1.1193	30.97	240	1.3803	0.3667
1.1193	32.0	248	1.2294	0.4167
1.1071	32.9	255	1.4134	0.5
1.1071	33.94	263	1.4123	0.3667
1.0429	34.97	271	1.2184	0.5
1.0528	36.0	279	1.3100	0.45
1.0528	36.9	286	1.3249	0.3833
1.0055	37.94	294	1.3051	0.5

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2