vit-base-patch16-224-dmae-va-U5-100-3i
This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.5087
- Accuracy: 0.8667
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 100
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy |
---|---|---|---|---|
No log | 0.9 | 7 | 0.5069 | 0.8333 |
0.3296 | 1.94 | 15 | 0.5087 | 0.8667 |
0.2919 | 2.97 | 23 | 0.5190 | 0.8667 |
0.2572 | 4.0 | 31 | 0.6483 | 0.7667 |
0.2572 | 4.9 | 38 | 0.5785 | 0.8167 |
0.2229 | 5.94 | 46 | 0.5932 | 0.8333 |
0.1799 | 6.97 | 54 | 0.5272 | 0.85 |
0.1563 | 8.0 | 62 | 0.6124 | 0.85 |
0.1563 | 8.9 | 69 | 0.6798 | 0.8167 |
0.125 | 9.94 | 77 | 0.7356 | 0.7833 |
0.1343 | 10.97 | 85 | 0.5086 | 0.85 |
0.0906 | 12.0 | 93 | 0.7601 | 0.7667 |
0.103 | 12.9 | 100 | 0.8084 | 0.8 |
0.103 | 13.94 | 108 | 0.5612 | 0.85 |
0.1002 | 14.97 | 116 | 0.6454 | 0.8333 |
0.1107 | 16.0 | 124 | 0.7783 | 0.8 |
0.1036 | 16.9 | 131 | 0.7857 | 0.7833 |
0.1036 | 17.94 | 139 | 0.6504 | 0.8167 |
0.1248 | 18.97 | 147 | 0.6510 | 0.8167 |
0.1074 | 20.0 | 155 | 0.7813 | 0.7833 |
0.1038 | 20.9 | 162 | 0.6553 | 0.8 |
0.1052 | 21.94 | 170 | 0.6449 | 0.8333 |
0.1052 | 22.97 | 178 | 0.7444 | 0.8 |
0.0782 | 24.0 | 186 | 1.0751 | 0.6833 |
0.0952 | 24.9 | 193 | 0.6453 | 0.8333 |
0.0803 | 25.94 | 201 | 0.7794 | 0.8 |
0.0803 | 26.97 | 209 | 0.6160 | 0.8333 |
0.0947 | 28.0 | 217 | 0.6362 | 0.85 |
0.0702 | 28.9 | 224 | 0.7610 | 0.8167 |
0.0737 | 29.94 | 232 | 0.7924 | 0.8167 |
0.0644 | 30.97 | 240 | 0.9755 | 0.8 |
0.0644 | 32.0 | 248 | 0.8580 | 0.8333 |
0.0695 | 32.9 | 255 | 1.1410 | 0.7167 |
0.09 | 33.94 | 263 | 0.8442 | 0.8 |
0.0619 | 34.97 | 271 | 1.1689 | 0.7167 |
0.0619 | 36.0 | 279 | 0.7599 | 0.8333 |
0.0607 | 36.9 | 286 | 0.8498 | 0.8167 |
0.0509 | 37.94 | 294 | 0.8331 | 0.85 |
0.0666 | 38.97 | 302 | 0.8166 | 0.8167 |
0.0615 | 40.0 | 310 | 0.9394 | 0.7667 |
0.0615 | 40.9 | 317 | 0.8837 | 0.8 |
0.0503 | 41.94 | 325 | 0.8208 | 0.8333 |
0.0431 | 42.97 | 333 | 1.1271 | 0.75 |
0.0548 | 44.0 | 341 | 0.9044 | 0.7833 |
0.0548 | 44.9 | 348 | 0.9017 | 0.8 |
0.0414 | 45.94 | 356 | 1.1390 | 0.75 |
0.0609 | 46.97 | 364 | 0.8937 | 0.8 |
0.0556 | 48.0 | 372 | 0.8459 | 0.8 |
0.0556 | 48.9 | 379 | 1.0285 | 0.7667 |
0.0417 | 49.94 | 387 | 0.7379 | 0.85 |
0.0409 | 50.97 | 395 | 0.7817 | 0.8333 |
0.0206 | 52.0 | 403 | 0.7860 | 0.8167 |
0.0414 | 52.9 | 410 | 0.8414 | 0.8167 |
0.0414 | 53.94 | 418 | 0.8657 | 0.8 |
0.0329 | 54.97 | 426 | 0.8824 | 0.8 |
0.0394 | 56.0 | 434 | 0.7990 | 0.8333 |
0.0373 | 56.9 | 441 | 0.8101 | 0.8167 |
0.0373 | 57.94 | 449 | 0.8535 | 0.8 |
0.0418 | 58.97 | 457 | 0.9149 | 0.8167 |
0.0365 | 60.0 | 465 | 0.9278 | 0.8 |
0.0367 | 60.9 | 472 | 0.9064 | 0.8 |
0.0355 | 61.94 | 480 | 0.9610 | 0.7833 |
0.0355 | 62.97 | 488 | 0.9174 | 0.8167 |
0.0492 | 64.0 | 496 | 0.9877 | 0.7667 |
0.0326 | 64.9 | 503 | 1.0192 | 0.7833 |
0.0233 | 65.94 | 511 | 0.9588 | 0.8 |
0.0233 | 66.97 | 519 | 0.9829 | 0.7833 |
0.0251 | 68.0 | 527 | 1.0540 | 0.7667 |
0.0283 | 68.9 | 534 | 1.0556 | 0.7667 |
0.0307 | 69.94 | 542 | 1.0036 | 0.7833 |
0.0319 | 70.97 | 550 | 0.9294 | 0.8 |
0.0319 | 72.0 | 558 | 1.0077 | 0.8 |
0.0246 | 72.9 | 565 | 1.0298 | 0.7833 |
0.0205 | 73.94 | 573 | 1.0041 | 0.7833 |
0.0345 | 74.97 | 581 | 0.9182 | 0.7833 |
0.0345 | 76.0 | 589 | 0.9054 | 0.8333 |
0.0181 | 76.9 | 596 | 0.9338 | 0.8333 |
0.0287 | 77.94 | 604 | 0.9678 | 0.7833 |
0.0268 | 78.97 | 612 | 0.9841 | 0.7833 |
0.0293 | 80.0 | 620 | 1.0380 | 0.7667 |
0.0293 | 80.9 | 627 | 1.0837 | 0.7833 |
0.0222 | 81.94 | 635 | 1.0132 | 0.7667 |
0.033 | 82.97 | 643 | 0.9785 | 0.8 |
0.0227 | 84.0 | 651 | 0.9848 | 0.8 |
0.0227 | 84.9 | 658 | 0.9780 | 0.8 |
0.0295 | 85.94 | 666 | 0.9613 | 0.8167 |
0.0291 | 86.97 | 674 | 0.9753 | 0.8167 |
0.031 | 88.0 | 682 | 0.9831 | 0.8 |
0.031 | 88.9 | 689 | 0.9820 | 0.8 |
0.0233 | 89.94 | 697 | 0.9793 | 0.8 |
0.0195 | 90.32 | 700 | 0.9788 | 0.8 |
Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0
- Downloads last month
- 9
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.