vit-beta2-0.9995
This model is a fine-tuned version of google/vit-base-patch16-224 on the skin-cancer dataset. It achieves the following results on the evaluation set:
- Loss: 0.5000
- Accuracy: 0.8519
- Precision: 0.8575
- Recall: 0.8519
- F1: 0.8527
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.9995) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1733
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 |
---|---|---|---|---|---|---|---|
1.7395 | 1.0 | 321 | 0.9629 | 0.6845 | 0.6040 | 0.6845 | 0.5948 |
1.1477 | 2.0 | 642 | 0.7630 | 0.7174 | 0.7438 | 0.7174 | 0.7138 |
1.0466 | 3.0 | 963 | 0.6211 | 0.7660 | 0.7543 | 0.7660 | 0.7373 |
0.9366 | 4.0 | 1284 | 0.6310 | 0.7354 | 0.7934 | 0.7354 | 0.7518 |
0.9629 | 5.0 | 1605 | 0.5584 | 0.7982 | 0.7934 | 0.7982 | 0.7865 |
0.9753 | 6.0 | 1926 | 0.5606 | 0.7920 | 0.7922 | 0.7920 | 0.7916 |
0.8629 | 7.0 | 2247 | 0.7154 | 0.6969 | 0.8040 | 0.6969 | 0.7238 |
0.7817 | 8.0 | 2568 | 0.6276 | 0.7597 | 0.8105 | 0.7597 | 0.7743 |
0.7355 | 9.0 | 2889 | 0.5259 | 0.8110 | 0.8215 | 0.8110 | 0.8133 |
0.6878 | 10.0 | 3210 | 0.5347 | 0.7885 | 0.8388 | 0.7885 | 0.8022 |
0.6206 | 11.0 | 3531 | 0.5543 | 0.7958 | 0.8375 | 0.7958 | 0.8079 |
0.5628 | 12.0 | 3852 | 0.6465 | 0.7819 | 0.8285 | 0.7819 | 0.7960 |
0.4607 | 13.0 | 4173 | 0.6070 | 0.8190 | 0.8411 | 0.8190 | 0.8195 |
0.4558 | 14.0 | 4494 | 0.6088 | 0.8138 | 0.8424 | 0.8138 | 0.8215 |
0.3984 | 15.0 | 4815 | 0.5507 | 0.8308 | 0.8486 | 0.8308 | 0.8343 |
0.3541 | 16.0 | 5136 | 0.5112 | 0.8412 | 0.8528 | 0.8412 | 0.8448 |
0.3203 | 17.0 | 5457 | 0.5000 | 0.8519 | 0.8575 | 0.8519 | 0.8527 |
0.2685 | 18.0 | 5778 | 0.5630 | 0.8429 | 0.8574 | 0.8429 | 0.8472 |
0.214 | 19.0 | 6099 | 0.5670 | 0.8405 | 0.8607 | 0.8405 | 0.8464 |
0.2195 | 20.0 | 6420 | 0.6209 | 0.8478 | 0.8557 | 0.8478 | 0.8500 |
0.1598 | 21.0 | 6741 | 0.5597 | 0.8630 | 0.8585 | 0.8630 | 0.8590 |
0.1174 | 22.0 | 7062 | 0.5682 | 0.8637 | 0.8617 | 0.8637 | 0.8624 |
0.1341 | 23.0 | 7383 | 0.6186 | 0.8662 | 0.8638 | 0.8662 | 0.8622 |
0.1307 | 24.0 | 7704 | 0.5961 | 0.8727 | 0.8710 | 0.8727 | 0.8701 |
0.1075 | 25.0 | 8025 | 0.5912 | 0.8738 | 0.8715 | 0.8738 | 0.8717 |
0.0741 | 26.0 | 8346 | 0.5991 | 0.8755 | 0.8711 | 0.8755 | 0.8713 |
0.1476 | 27.0 | 8667 | 0.5900 | 0.8745 | 0.8705 | 0.8745 | 0.8714 |
Framework versions
- Transformers 4.40.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 17