jaypratap's picture
End of training
b52facb verified
metadata
license: apache-2.0
base_model: google/vit-base-patch16-224
tags:
  - masked-auto-encoding
  - generated_from_trainer
datasets:
  - imagefolder
model-index:
  - name: vit-pretraining-2024_03_10
    results: []

vit-pretraining-2024_03_10

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4444

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 4.6875e-06
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 200.0

Training results

Training Loss Epoch Step Validation Loss
1.0002 1.0 2443 1.0000
0.9832 2.0 4886 0.9753
0.9246 3.0 7329 0.9304
0.8979 4.0 9772 0.8855
0.8307 5.0 12215 0.8077
0.7861 6.0 14658 0.7776
0.7665 7.0 17101 0.7557
0.7421 8.0 19544 0.7337
0.6841 9.0 21987 0.7133
0.6875 10.0 24430 0.7001
0.6991 11.0 26873 0.6887
0.6991 12.0 29316 0.6711
0.6584 13.0 31759 0.6674
0.6619 14.0 34202 0.6507
0.6389 15.0 36645 0.6462
0.6381 16.0 39088 0.6370
0.616 17.0 41531 0.6248
0.627 18.0 43974 0.6213
0.6179 19.0 46417 0.6150
0.6226 20.0 48860 0.6112
0.5876 21.0 51303 0.6062
0.613 22.0 53746 0.5990
0.5864 23.0 56189 0.5948
0.5741 24.0 58632 0.5940
0.5886 25.0 61075 0.5883
0.6028 26.0 63518 0.5890
0.578 27.0 65961 0.5841
0.5846 28.0 68404 0.5779
0.5725 29.0 70847 0.5766
0.5684 30.0 73290 0.5791
0.5689 31.0 75733 0.5726
0.5478 32.0 78176 0.5708
0.5739 33.0 80619 0.5697
0.5578 34.0 83062 0.5629
0.568 35.0 85505 0.5696
0.5819 36.0 87948 0.5649
0.5442 37.0 90391 0.5649
0.5616 38.0 92834 0.5626
0.5386 39.0 95277 0.5617
0.5725 40.0 97720 0.5552
0.549 41.0 100163 0.5621
0.5539 42.0 102606 0.5535
0.5513 43.0 105049 0.5514
0.5538 44.0 107492 0.5480
0.5423 45.0 109935 0.5488
0.5431 46.0 112378 0.5466
0.5495 47.0 114821 0.5442
0.5593 48.0 117264 0.5447
0.5488 49.0 119707 0.5431
0.5203 50.0 122150 0.5391
0.5386 51.0 124593 0.5384
0.5498 52.0 127036 0.5393
0.5391 53.0 129479 0.5372
0.5361 54.0 131922 0.5363
0.5295 55.0 134365 0.5343
0.5227 56.0 136808 0.5345
0.5182 57.0 139251 0.5287
0.5103 58.0 141694 0.5303
0.5411 59.0 144137 0.5278
0.5187 60.0 146580 0.5259
0.5272 61.0 149023 0.5254
0.5352 62.0 151466 0.5264
0.5243 63.0 153909 0.5214
0.5134 64.0 156352 0.5210
0.5305 65.0 158795 0.5238
0.5507 66.0 161238 0.5210
0.5179 67.0 163681 0.5217
0.5162 68.0 166124 0.5166
0.5192 69.0 168567 0.5201
0.5231 70.0 171010 0.5175
0.5095 71.0 173453 0.5138
0.5205 72.0 175896 0.5135
0.5299 73.0 178339 0.5147
0.4947 74.0 180782 0.5112
0.5133 75.0 183225 0.5115
0.4886 76.0 185668 0.5090
0.5288 77.0 188111 0.5105
0.514 78.0 190554 0.5072
0.4803 79.0 192997 0.5053
0.4882 80.0 195440 0.5075
0.5037 81.0 197883 0.5063
0.5314 82.0 200326 0.5027
0.5181 83.0 202769 0.5013
0.5191 84.0 205212 0.5009
0.503 85.0 207655 0.4980
0.4894 86.0 210098 0.4993
0.4801 87.0 212541 0.4964
0.5019 88.0 214984 0.4956
0.5036 89.0 217427 0.4927
0.4844 90.0 219870 0.4932
0.4656 91.0 222313 0.4890
0.4839 92.0 224756 0.4881
0.4955 93.0 227199 0.4880
0.4792 94.0 229642 0.4877
0.4655 95.0 232085 0.4833
0.4811 96.0 234528 0.4835
0.5118 97.0 236971 0.4842
0.479 98.0 239414 0.4830
0.4693 99.0 241857 0.4827
0.46 100.0 244300 0.4785
0.479 101.0 246743 0.4792
0.4702 102.0 249186 0.4793
0.4683 103.0 251629 0.4757
0.4682 104.0 254072 0.4750
0.4749 105.0 256515 0.4747
0.4915 106.0 258958 0.4719
0.4832 107.0 261401 0.4729
0.4371 108.0 263844 0.4720
0.4779 109.0 266287 0.4710
0.4796 110.0 268730 0.4693
0.463 111.0 271173 0.4696
0.4722 112.0 273616 0.4679
0.4689 113.0 276059 0.4693
0.4644 114.0 278502 0.4665
0.4688 115.0 280945 0.4674
0.4619 116.0 283388 0.4644
0.4533 117.0 285831 0.4663
0.4604 118.0 288274 0.4634
0.4722 119.0 290717 0.4637
0.4622 120.0 293160 0.4634
0.4575 121.0 295603 0.4628
0.4824 122.0 298046 0.4631
0.4757 123.0 300489 0.4620
0.4457 124.0 302932 0.4620
0.4471 125.0 305375 0.4599
0.444 126.0 307818 0.4575
0.4521 127.0 310261 0.4599
0.4441 128.0 312704 0.4588
0.4432 129.0 315147 0.4596
0.4518 130.0 317590 0.4550
0.4457 131.0 320033 0.4578
0.4529 132.0 322476 0.4543
0.4871 133.0 324919 0.4560
0.4482 134.0 327362 0.4546
0.4648 135.0 329805 0.4574
0.4372 136.0 332248 0.4546
0.4353 137.0 334691 0.4531
0.4446 138.0 337134 0.4539
0.4666 139.0 339577 0.4518
0.4734 140.0 342020 0.4528
0.4601 141.0 344463 0.4540
0.4415 142.0 346906 0.4528
0.459 143.0 349349 0.4505
0.454 144.0 351792 0.4514
0.4606 145.0 354235 0.4511
0.4315 146.0 356678 0.4514
0.4583 147.0 359121 0.4520
0.452 148.0 361564 0.4495
0.4449 149.0 364007 0.4508
0.4272 150.0 366450 0.4489
0.439 151.0 368893 0.4504
0.4586 152.0 371336 0.4503
0.4559 153.0 373779 0.4500
0.4527 154.0 376222 0.4492
0.4511 155.0 378665 0.4491
0.4405 156.0 381108 0.4488
0.4509 157.0 383551 0.4482
0.4713 158.0 385994 0.4480
0.4578 159.0 388437 0.4465
0.4154 160.0 390880 0.4464
0.4399 161.0 393323 0.4488
0.4547 162.0 395766 0.4476
0.4426 163.0 398209 0.4456
0.4517 164.0 400652 0.4484
0.4376 165.0 403095 0.4455
0.4463 166.0 405538 0.4463
0.4289 167.0 407981 0.4466
0.4291 168.0 410424 0.4469
0.4623 169.0 412867 0.4455
0.4673 170.0 415310 0.4455
0.4609 171.0 417753 0.4456
0.4478 172.0 420196 0.4468
0.4521 173.0 422639 0.4437
0.4378 174.0 425082 0.4460
0.4361 175.0 427525 0.4446
0.4321 176.0 429968 0.4451
0.4369 177.0 432411 0.4451
0.4381 178.0 434854 0.4443
0.4408 179.0 437297 0.4449
0.4414 180.0 439740 0.4448
0.4333 181.0 442183 0.4438
0.4468 182.0 444626 0.4452
0.4394 183.0 447069 0.4440
0.441 184.0 449512 0.4434
0.4546 185.0 451955 0.4462
0.4455 186.0 454398 0.4458
0.4431 187.0 456841 0.4426
0.4489 188.0 459284 0.4433
0.4485 189.0 461727 0.4435
0.4449 190.0 464170 0.4433
0.4482 191.0 466613 0.4449
0.4395 192.0 469056 0.4433
0.4557 193.0 471499 0.4436
0.4208 194.0 473942 0.4450
0.4274 195.0 476385 0.4429
0.4423 196.0 478828 0.4434
0.4331 197.0 481271 0.4453
0.43 198.0 483714 0.4448
0.4308 199.0 486157 0.4460
0.4373 200.0 488600 0.4430

Framework versions

  • Transformers 4.39.0.dev0
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2