vit-base-patch16-224-dmae-va-U5-42B

This model is a fine-tuned version of google/vit-base-patch16-224 on Augusto777/dmae-ve-U5 dataset. It achieves the following results on the evaluation set:

• Loss: 0.7215
• Accuracy: 0.85

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.3101	0.4667
1.408	1.94	15	1.1884	0.4833
1.1286	2.97	23	0.9476	0.5167
0.7589	4.0	31	0.7637	0.75
0.7589	4.9	38	0.7186	0.6833
0.4786	5.94	46	0.6192	0.7833
0.2874	6.97	54	0.6195	0.7833
0.2027	8.0	62	0.5959	0.7833
0.2027	8.9	69	0.6104	0.7667
0.1662	9.94	77	0.7297	0.75
0.1462	10.97	85	0.7852	0.7667
0.1419	12.0	93	0.8637	0.7167
0.1199	12.9	100	0.6797	0.7333
0.1199	13.94	108	0.7660	0.7667
0.0949	14.97	116	0.7386	0.7167
0.0901	16.0	124	1.0126	0.7
0.0808	16.9	131	0.7060	0.8
0.0808	17.94	139	0.7857	0.7833
0.102	18.97	147	0.7411	0.8
0.0706	20.0	155	0.7340	0.8167
0.0582	20.9	162	0.8589	0.75
0.0687	21.94	170	0.8546	0.7667
0.0687	22.97	178	0.7761	0.7667
0.0633	24.0	186	0.8112	0.7667
0.0626	24.9	193	0.6943	0.8333
0.0578	25.94	201	0.8593	0.7833
0.0578	26.97	209	0.7215	0.85
0.0434	28.0	217	0.8150	0.8
0.0492	28.9	224	0.7834	0.7833
0.0582	29.94	232	0.7844	0.7833
0.0515	30.97	240	0.7973	0.7667
0.0515	32.0	248	0.7744	0.8
0.0487	32.9	255	0.8614	0.75
0.0455	33.94	263	0.8195	0.7667
0.0329	34.97	271	0.8327	0.7667
0.0329	36.0	279	0.8889	0.7667
0.0447	36.9	286	0.8705	0.7667
0.0445	37.94	294	0.8695	0.7667

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2