vit-base-patch16-224-dmae-va-U4-40X

This model is a fine-tuned version of google/vit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9065
• Accuracy: 0.7843

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.3427	0.3529
1.445	1.94	15	1.2622	0.4314
1.445	2.97	23	0.9845	0.6667
1.0925	4.0	31	0.8044	0.6667
0.6396	4.9	38	0.7464	0.7059
0.6396	5.94	46	0.7067	0.6863
0.396	6.97	54	0.6942	0.7451
0.2346	8.0	62	0.7595	0.7059
0.2346	8.9	69	0.7097	0.7451
0.1738	9.94	77	0.7498	0.7059
0.1487	10.97	85	0.8513	0.6667
0.1487	12.0	93	0.8439	0.7647
0.1295	12.9	100	0.7515	0.7059
0.1088	13.94	108	0.9065	0.7843
0.1088	14.97	116	0.7766	0.7451
0.0914	16.0	124	0.9380	0.6863
0.0914	16.9	131	0.8405	0.7451
0.0892	17.94	139	0.9231	0.7255
0.0995	18.97	147	0.8269	0.7255
0.0995	20.0	155	1.0265	0.7059
0.0917	20.9	162	0.8303	0.7059
0.0942	21.94	170	0.8248	0.7451
0.0942	22.97	178	0.8935	0.7451
0.0697	24.0	186	0.8769	0.7451
0.082	24.9	193	0.8742	0.7451
0.082	25.94	201	1.1143	0.7059
0.0654	26.97	209	0.8933	0.7059
0.0485	28.0	217	0.9003	0.7451
0.0485	28.9	224	1.0644	0.7059
0.0741	29.94	232	1.0052	0.7059
0.0442	30.97	240	0.9812	0.7843
0.0442	32.0	248	0.9723	0.7451
0.0596	32.9	255	0.9812	0.7451
0.0596	33.94	263	0.9352	0.7647
0.0656	34.97	271	0.9575	0.7647
0.0423	36.0	279	0.9781	0.7647
0.0423	36.13	280	0.9783	0.7647

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2