Edit model card

deit-base-distilled-patch16-224-55-fold2

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5842
  • Accuracy: 0.8101

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 0.8571 3 0.7763 0.5443
No log 2.0 7 0.6780 0.6203
0.721 2.8571 10 0.6954 0.5316
0.721 4.0 14 0.6370 0.6203
0.721 4.8571 17 0.6105 0.5949
0.6207 6.0 21 0.5798 0.6835
0.6207 6.8571 24 0.5704 0.7468
0.6207 8.0 28 0.5879 0.7089
0.5427 8.8571 31 0.6727 0.6582
0.5427 10.0 35 0.5841 0.6962
0.5427 10.8571 38 0.6059 0.6962
0.4775 12.0 42 1.0271 0.6076
0.4775 12.8571 45 0.6412 0.7089
0.4775 14.0 49 0.8064 0.6582
0.4961 14.8571 52 0.5600 0.6582
0.4961 16.0 56 0.5889 0.6709
0.4961 16.8571 59 0.8381 0.6835
0.4391 18.0 63 0.6725 0.6962
0.4391 18.8571 66 0.5350 0.7215
0.413 20.0 70 0.6033 0.7089
0.413 20.8571 73 0.7280 0.6835
0.413 22.0 77 0.6082 0.7342
0.336 22.8571 80 0.6530 0.7595
0.336 24.0 84 0.6922 0.7089
0.336 24.8571 87 0.6649 0.7089
0.2745 26.0 91 0.7311 0.7089
0.2745 26.8571 94 0.7192 0.7089
0.2745 28.0 98 0.7408 0.7215
0.2494 28.8571 101 0.5842 0.8101
0.2494 30.0 105 0.5949 0.7595
0.2494 30.8571 108 0.6885 0.7468
0.2291 32.0 112 0.8746 0.7089
0.2291 32.8571 115 0.8005 0.7089
0.2291 34.0 119 0.7034 0.7342
0.2 34.8571 122 0.7047 0.7089
0.2 36.0 126 0.8362 0.7342
0.2 36.8571 129 0.8509 0.7468
0.1674 38.0 133 0.9237 0.7595
0.1674 38.8571 136 0.7527 0.7595
0.1764 40.0 140 0.7904 0.7468
0.1764 40.8571 143 0.7333 0.7595
0.1764 42.0 147 0.7778 0.7342
0.1706 42.8571 150 0.7342 0.7722
0.1706 44.0 154 0.8144 0.7468
0.1706 44.8571 157 0.8299 0.7595
0.1617 46.0 161 1.0111 0.7468
0.1617 46.8571 164 0.8602 0.7595
0.1617 48.0 168 0.8332 0.7342
0.1622 48.8571 171 0.8297 0.7468
0.1622 50.0 175 0.8817 0.7595
0.1622 50.8571 178 0.8742 0.7595
0.1437 52.0 182 1.0696 0.7595
0.1437 52.8571 185 0.9412 0.7595
0.1437 54.0 189 0.7411 0.7975
0.1492 54.8571 192 0.9043 0.7595
0.1492 56.0 196 0.7936 0.7848
0.1492 56.8571 199 0.8231 0.7722
0.1279 58.0 203 1.0894 0.7722
0.1279 58.8571 206 1.0071 0.7975
0.1317 60.0 210 0.9893 0.7722
0.1317 60.8571 213 1.0476 0.7468
0.1317 62.0 217 0.8081 0.7848
0.1456 62.8571 220 0.8136 0.7468
0.1456 64.0 224 0.9613 0.7848
0.1456 64.8571 227 0.9783 0.7848
0.119 66.0 231 1.0226 0.7722
0.119 66.8571 234 1.0810 0.7722
0.119 68.0 238 0.9606 0.7975
0.1323 68.8571 241 0.9852 0.7848
0.1323 70.0 245 0.8826 0.7595
0.1323 70.8571 248 0.8169 0.7468
0.126 72.0 252 0.8815 0.7595
0.126 72.8571 255 0.9871 0.7722
0.126 74.0 259 0.8927 0.7722
0.1013 74.8571 262 0.8365 0.7468
0.1013 76.0 266 0.8423 0.7468
0.1013 76.8571 269 0.8331 0.7468
0.1142 78.0 273 0.8204 0.7722
0.1142 78.8571 276 0.8286 0.7722
0.1287 80.0 280 0.8702 0.7722
0.1287 80.8571 283 0.9070 0.7722
0.1287 82.0 287 0.9025 0.7722
0.099 82.8571 290 0.8806 0.7722
0.099 84.0 294 0.8637 0.7595
0.099 84.8571 297 0.8578 0.7595
0.1141 85.7143 300 0.8551 0.7468

Framework versions

  • Transformers 4.41.0
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
2
Safetensors
Model size
85.8M params
Tensor type
F32
·

Finetuned from

Evaluation results