vit-beta1-0.95
This model is a fine-tuned version of google/vit-base-patch16-224 on the skin-cancer dataset. It achieves the following results on the evaluation set:
- Loss: 0.5187
- Accuracy: 0.8384
- Precision: 0.8496
- Recall: 0.8384
- F1: 0.8421
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.95,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1733
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 |
---|---|---|---|---|---|---|---|
1.7463 | 1.0 | 321 | 1.0838 | 0.6810 | 0.6594 | 0.6810 | 0.6416 |
1.2057 | 2.0 | 642 | 0.7826 | 0.7025 | 0.7172 | 0.7025 | 0.6909 |
1.0261 | 3.0 | 963 | 0.7040 | 0.7406 | 0.7119 | 0.7406 | 0.6962 |
1.0269 | 4.0 | 1284 | 0.6113 | 0.7573 | 0.7934 | 0.7573 | 0.7620 |
0.943 | 5.0 | 1605 | 0.7935 | 0.6848 | 0.7925 | 0.6848 | 0.7063 |
0.9677 | 6.0 | 1926 | 0.6434 | 0.7555 | 0.7996 | 0.7555 | 0.7690 |
0.875 | 7.0 | 2247 | 0.6981 | 0.7157 | 0.7962 | 0.7157 | 0.7344 |
0.7796 | 8.0 | 2568 | 0.7443 | 0.7115 | 0.8097 | 0.7115 | 0.7352 |
0.7308 | 9.0 | 2889 | 0.5311 | 0.8103 | 0.8217 | 0.8103 | 0.8125 |
0.7036 | 10.0 | 3210 | 0.5371 | 0.7892 | 0.8341 | 0.7892 | 0.8006 |
0.637 | 11.0 | 3531 | 0.6372 | 0.7850 | 0.8268 | 0.7850 | 0.7963 |
0.5668 | 12.0 | 3852 | 0.5850 | 0.8024 | 0.8279 | 0.8024 | 0.8109 |
0.4823 | 13.0 | 4173 | 0.6345 | 0.8284 | 0.8341 | 0.8284 | 0.8115 |
0.4808 | 14.0 | 4494 | 0.6609 | 0.7992 | 0.8328 | 0.7992 | 0.8038 |
0.4087 | 15.0 | 4815 | 0.6568 | 0.7791 | 0.8373 | 0.7791 | 0.7952 |
0.3579 | 16.0 | 5136 | 0.5462 | 0.8294 | 0.8510 | 0.8294 | 0.8364 |
0.3015 | 17.0 | 5457 | 0.5187 | 0.8384 | 0.8496 | 0.8384 | 0.8421 |
0.2794 | 18.0 | 5778 | 0.6125 | 0.8204 | 0.8538 | 0.8204 | 0.8300 |
0.2297 | 19.0 | 6099 | 0.5672 | 0.8391 | 0.8500 | 0.8391 | 0.8429 |
0.229 | 20.0 | 6420 | 0.6051 | 0.8526 | 0.8543 | 0.8526 | 0.8524 |
0.1743 | 21.0 | 6741 | 0.5990 | 0.8606 | 0.8598 | 0.8606 | 0.8577 |
0.1327 | 22.0 | 7062 | 0.6290 | 0.8568 | 0.8636 | 0.8568 | 0.8590 |
0.1321 | 23.0 | 7383 | 0.6259 | 0.8655 | 0.8613 | 0.8655 | 0.8625 |
0.1236 | 24.0 | 7704 | 0.6135 | 0.8703 | 0.8699 | 0.8703 | 0.8697 |
0.126 | 25.0 | 8025 | 0.6107 | 0.8655 | 0.8658 | 0.8655 | 0.8645 |
0.0919 | 26.0 | 8346 | 0.6001 | 0.8700 | 0.8657 | 0.8700 | 0.8665 |
0.1421 | 27.0 | 8667 | 0.5894 | 0.8745 | 0.8721 | 0.8745 | 0.8727 |
Framework versions
- Transformers 4.40.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 2