vit-dropout-0.5
This model is a fine-tuned version of google/vit-base-patch16-224 on the skin-cancer dataset. It achieves the following results on the evaluation set:
- Loss: 0.4899
- Accuracy: 0.8592
- Precision: 0.8593
- Recall: 0.8592
- F1: 0.8581
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1219
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 |
---|---|---|---|---|---|---|---|
1.6642 | 1.0 | 321 | 0.8597 | 0.7178 | 0.6898 | 0.7178 | 0.6753 |
1.1191 | 2.0 | 642 | 0.6286 | 0.7656 | 0.7678 | 0.7656 | 0.7653 |
1.0741 | 3.0 | 963 | 0.6771 | 0.7545 | 0.7795 | 0.7545 | 0.7353 |
1.0023 | 4.0 | 1284 | 0.6269 | 0.7708 | 0.7839 | 0.7708 | 0.7727 |
0.92 | 5.0 | 1605 | 0.6010 | 0.7816 | 0.7913 | 0.7816 | 0.7811 |
0.9155 | 6.0 | 1926 | 0.5399 | 0.7944 | 0.8035 | 0.7944 | 0.7979 |
0.7971 | 7.0 | 2247 | 0.7016 | 0.7347 | 0.7903 | 0.7347 | 0.7509 |
0.7171 | 8.0 | 2568 | 0.6352 | 0.7396 | 0.8175 | 0.7396 | 0.7601 |
0.676 | 9.0 | 2889 | 0.5073 | 0.8072 | 0.8241 | 0.8072 | 0.8120 |
0.6312 | 10.0 | 3210 | 0.5552 | 0.7850 | 0.8374 | 0.7850 | 0.7978 |
0.5777 | 11.0 | 3531 | 0.5175 | 0.8173 | 0.8428 | 0.8173 | 0.8247 |
0.4686 | 12.0 | 3852 | 0.5712 | 0.8062 | 0.8413 | 0.8062 | 0.8172 |
0.4134 | 13.0 | 4173 | 0.6331 | 0.8214 | 0.8384 | 0.8214 | 0.8180 |
0.3904 | 14.0 | 4494 | 0.6161 | 0.8308 | 0.8546 | 0.8308 | 0.8357 |
0.3388 | 15.0 | 4815 | 0.5141 | 0.8384 | 0.8519 | 0.8384 | 0.8425 |
0.301 | 16.0 | 5136 | 0.5525 | 0.8360 | 0.8521 | 0.8360 | 0.8416 |
0.2516 | 17.0 | 5457 | 0.4899 | 0.8592 | 0.8593 | 0.8592 | 0.8581 |
0.2433 | 18.0 | 5778 | 0.5183 | 0.8509 | 0.8565 | 0.8509 | 0.8528 |
0.1769 | 19.0 | 6099 | 0.6016 | 0.8426 | 0.8622 | 0.8426 | 0.8487 |
0.179 | 20.0 | 6420 | 0.6330 | 0.8551 | 0.8543 | 0.8551 | 0.8534 |
0.1401 | 21.0 | 6741 | 0.5884 | 0.8669 | 0.8612 | 0.8669 | 0.8630 |
0.1085 | 22.0 | 7062 | 0.6131 | 0.8627 | 0.8591 | 0.8627 | 0.8603 |
0.1188 | 23.0 | 7383 | 0.6344 | 0.8648 | 0.8581 | 0.8648 | 0.8596 |
0.106 | 24.0 | 7704 | 0.6152 | 0.8731 | 0.8687 | 0.8731 | 0.8696 |
0.1029 | 25.0 | 8025 | 0.6209 | 0.8717 | 0.8676 | 0.8717 | 0.8692 |
0.0761 | 26.0 | 8346 | 0.6190 | 0.8727 | 0.8679 | 0.8727 | 0.8694 |
0.1207 | 27.0 | 8667 | 0.6154 | 0.8721 | 0.8677 | 0.8721 | 0.8693 |
Framework versions
- Transformers 4.40.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 4