vit-base-patch16-224-dmae-va-U5-40

This model is a fine-tuned version of google/vit-base-patch16-224 on Augusto777/dmae-ve-U5 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0367
• Accuracy: 0.8166

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.4236	0.3605
1.3165	2.0	13	1.1072	0.5306
1.3165	2.92	19	0.9370	0.6463
0.93	4.0	26	0.6870	0.7687
0.93	4.92	32	0.4743	0.8163
0.5368	6.0	39	0.2825	0.9184
0.5368	6.92	45	0.2066	0.9524
0.2989	8.0	52	0.1224	0.9728
0.2989	8.92	58	0.1453	0.9592
0.1746	10.0	65	0.0367	1.0
0.1746	10.92	71	0.0616	0.9864
0.1596	12.0	78	0.0234	1.0
0.1094	12.92	84	0.0298	1.0
0.1094	14.0	91	0.0444	0.9932
0.1123	14.92	97	0.0251	1.0
0.1123	16.0	104	0.0185	1.0
0.0761	16.92	110	0.0159	1.0
0.0761	18.0	117	0.0180	1.0
0.0743	18.92	123	0.0111	1.0
0.0743	20.0	130	0.0134	1.0
0.072	20.92	136	0.0123	1.0
0.072	22.0	143	0.0100	1.0
0.0744	22.92	149	0.0074	1.0
0.0688	24.0	156	0.0064	1.0
0.0688	24.92	162	0.0070	1.0
0.0737	26.0	169	0.0064	1.0
0.0737	26.92	175	0.0055	1.0
0.053	28.0	182	0.0075	1.0
0.053	28.92	188	0.0046	1.0
0.0677	30.0	195	0.0046	1.0
0.0677	30.92	201	0.0098	1.0
0.055	32.0	208	0.0085	1.0
0.055	32.92	214	0.0056	1.0
0.0576	34.0	221	0.0059	1.0
0.0576	34.92	227	0.0054	1.0
0.0697	36.0	234	0.0055	1.0
0.0415	36.92	240	0.0056	1.0

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2