SL-CvT

This model is a fine-tuned version of microsoft/cvt-13 on the imagefolder dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3430
  • F1: 0.9298
  • Roc Auc: 0.9777
  • Accuracy: 0.9317

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss F1 Roc Auc Accuracy
1.2379 1.0 60 1.0716 0.6422 0.7323 0.7246
1.0186 2.0 120 0.8477 0.6425 0.7879 0.7293
0.9433 3.0 180 0.7473 0.7060 0.8454 0.7538
0.8644 4.0 240 0.6831 0.7188 0.8696 0.7663
0.7985 5.0 300 0.6420 0.7409 0.8943 0.7799
0.7322 6.0 360 0.5713 0.7886 0.9196 0.8101
0.725 7.0 420 0.5311 0.7989 0.9324 0.8190
0.6529 8.0 480 0.5246 0.7852 0.9404 0.8117
0.6224 9.0 540 0.4598 0.8282 0.9517 0.8440
0.6315 10.0 600 0.4363 0.8457 0.9585 0.8529
0.5651 11.0 660 0.4437 0.8323 0.9564 0.8503
0.574 12.0 720 0.4003 0.8531 0.9617 0.8638
0.5269 13.0 780 0.3901 0.8676 0.9671 0.8722
0.5138 14.0 840 0.3984 0.8607 0.9685 0.8732
0.4839 15.0 900 0.3763 0.8683 0.9701 0.8769
0.463 16.0 960 0.3398 0.8837 0.9718 0.8894
0.4767 17.0 1020 0.3293 0.8846 0.9738 0.8915
0.4985 18.0 1080 0.3350 0.8852 0.9763 0.8863
0.4657 19.0 1140 0.3369 0.8872 0.9746 0.8951
0.4514 20.0 1200 0.3213 0.8880 0.9750 0.8925
0.4207 21.0 1260 0.3175 0.8943 0.9771 0.8978
0.4522 22.0 1320 0.3229 0.8970 0.9767 0.8983
0.4328 23.0 1380 0.3121 0.8948 0.9791 0.8978
0.3942 24.0 1440 0.3111 0.8993 0.9765 0.9030
0.4414 25.0 1500 0.3062 0.9032 0.9763 0.9061
0.3608 26.0 1560 0.3099 0.8997 0.9787 0.9014
0.3729 27.0 1620 0.3050 0.9029 0.9783 0.9082
0.393 28.0 1680 0.2970 0.9090 0.9797 0.9108
0.402 29.0 1740 0.2986 0.9087 0.9793 0.9113
0.3697 30.0 1800 0.3384 0.8968 0.9769 0.9025
0.3502 31.0 1860 0.3035 0.9058 0.9789 0.9103
0.3653 32.0 1920 0.3127 0.9024 0.9788 0.9025
0.3898 33.0 1980 0.3222 0.9050 0.9778 0.9061
0.317 34.0 2040 0.3013 0.9124 0.9798 0.9139
0.3166 35.0 2100 0.3185 0.9095 0.9775 0.9134
0.3771 36.0 2160 0.3067 0.9049 0.9782 0.9066
0.3487 37.0 2220 0.2948 0.9118 0.9801 0.9134
0.3202 38.0 2280 0.2916 0.9168 0.9788 0.9186
0.3163 39.0 2340 0.3149 0.9141 0.9777 0.9155
0.3605 40.0 2400 0.2964 0.9192 0.9797 0.9207
0.3636 41.0 2460 0.3142 0.9111 0.9810 0.9134
0.3454 42.0 2520 0.3133 0.9111 0.9792 0.9113
0.3561 43.0 2580 0.3090 0.9073 0.9804 0.9077
0.3136 44.0 2640 0.3236 0.9144 0.9782 0.9176
0.3529 45.0 2700 0.3054 0.9175 0.9800 0.9202
0.2987 46.0 2760 0.2944 0.9222 0.9802 0.9233
0.2966 47.0 2820 0.3215 0.9201 0.9786 0.9233
0.3203 48.0 2880 0.3150 0.9219 0.9797 0.9244
0.2821 49.0 2940 0.3072 0.9273 0.9800 0.9291
0.2852 50.0 3000 0.3265 0.9155 0.9792 0.9176
0.3544 51.0 3060 0.3175 0.9150 0.9802 0.9150
0.3327 52.0 3120 0.3134 0.9222 0.9802 0.9244
0.2877 53.0 3180 0.3222 0.9154 0.9805 0.9165
0.3089 54.0 3240 0.3045 0.9248 0.9811 0.9259
0.2904 55.0 3300 0.3301 0.9175 0.9787 0.9186
0.2821 56.0 3360 0.3069 0.9206 0.9810 0.9218
0.321 57.0 3420 0.3209 0.9254 0.9800 0.9270
0.2995 58.0 3480 0.3281 0.9202 0.9802 0.9233
0.2683 59.0 3540 0.3263 0.9174 0.9802 0.9202
0.3021 60.0 3600 0.3484 0.9170 0.9788 0.9186
0.3262 61.0 3660 0.3270 0.9151 0.9807 0.9165
0.2329 62.0 3720 0.3280 0.9211 0.9807 0.9233
0.2935 63.0 3780 0.3296 0.9244 0.9807 0.9264
0.2856 64.0 3840 0.3323 0.9209 0.9811 0.9218
0.2829 65.0 3900 0.3390 0.9200 0.9802 0.9218
0.3044 66.0 3960 0.3324 0.9215 0.9799 0.9228
0.2767 67.0 4020 0.3496 0.9150 0.9778 0.9160
0.2936 68.0 4080 0.3378 0.9257 0.9790 0.9275
0.2884 69.0 4140 0.3493 0.9227 0.9790 0.9249
0.2906 70.0 4200 0.3408 0.9259 0.9794 0.9275
0.2542 71.0 4260 0.3559 0.9233 0.9769 0.9249
0.2557 72.0 4320 0.3481 0.9237 0.9779 0.9254
0.2266 73.0 4380 0.3518 0.9208 0.9781 0.9223
0.2771 74.0 4440 0.3544 0.9231 0.9776 0.9254
0.2747 75.0 4500 0.3469 0.9270 0.9780 0.9285
0.2443 76.0 4560 0.3513 0.9216 0.9767 0.9233
0.2859 77.0 4620 0.3456 0.9234 0.9771 0.9254
0.2677 78.0 4680 0.3474 0.9239 0.9780 0.9254
0.2492 79.0 4740 0.3513 0.9235 0.9778 0.9254
0.2532 80.0 4800 0.3524 0.9210 0.9773 0.9233
0.2646 81.0 4860 0.3529 0.9240 0.9784 0.9238
0.2842 82.0 4920 0.3433 0.9260 0.9777 0.9280
0.2872 83.0 4980 0.3584 0.9272 0.9771 0.9285
0.2678 84.0 5040 0.3430 0.9298 0.9777 0.9317
0.2705 85.0 5100 0.3534 0.9268 0.9777 0.9291
0.2605 86.0 5160 0.3574 0.9272 0.9777 0.9296
0.2572 87.0 5220 0.3426 0.9273 0.9781 0.9291
0.2646 88.0 5280 0.3472 0.9234 0.9789 0.9244
0.2831 89.0 5340 0.3433 0.9272 0.9779 0.9291
0.277 90.0 5400 0.3441 0.9263 0.9789 0.9280
0.2584 91.0 5460 0.3432 0.9236 0.9788 0.9249
0.2703 92.0 5520 0.3409 0.9248 0.9789 0.9259
0.2811 93.0 5580 0.3449 0.9215 0.9795 0.9228
0.2786 94.0 5640 0.3465 0.9260 0.9789 0.9280
0.267 95.0 5700 0.3472 0.9260 0.9791 0.9275
0.2695 96.0 5760 0.3500 0.9268 0.9786 0.9285
0.279 97.0 5820 0.3582 0.9249 0.9782 0.9270
0.2774 98.0 5880 0.3486 0.9251 0.9790 0.9270
0.2512 99.0 5940 0.3514 0.9287 0.9786 0.9306
0.2218 100.0 6000 0.3482 0.9269 0.9789 0.9285

Framework versions

  • Transformers 4.29.2
  • Pytorch 2.0.0+cu118
  • Datasets 2.12.0
  • Tokenizers 0.13.3
Downloads last month
20
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Evaluation results