--- language: - eng license: apache-2.0 base_model: facebook/dinov2-large tags: - multilabel-image-classification - multilabel - generated_from_trainer metrics: - accuracy model-index: - name: dinov2-large-2024_01_24-with_data_aug_batch-size32_epochs85_freeze results: [] --- # dinov2-large-2024_01_24-with_data_aug_batch-size32_epochs85_freeze This model is a fine-tuned version of [facebook/dinov2-large](https://huggingface.co/facebook/dinov2-large) on the multilabel_complete_dataset dataset. It achieves the following results on the evaluation set: - Loss: 0.0864 - F1 Micro: 0.8668 - F1 Macro: 0.8381 - Roc Auc: 0.9138 - Accuracy: 0.5805 - Learning Rate: 0.0000 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 32 - eval_batch_size: 32 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 85 ### Training results | Training Loss | Epoch | Step | Accuracy | F1 Macro | F1 Micro | Validation Loss | Roc Auc | Rate | |:-------------:|:-----:|:-----:|:--------:|:--------:|:--------:|:---------------:|:-------:|:------:| | No log | 1.0 | 274 | 0.4589 | 0.6395 | 0.7738 | 0.1359 | 0.8471 | 0.001 | | 0.2459 | 2.0 | 548 | 0.4941 | 0.7305 | 0.8032 | 0.1236 | 0.8697 | 0.001 | | 0.2459 | 3.0 | 822 | 0.5125 | 0.7426 | 0.8174 | 0.1167 | 0.8828 | 0.001 | | 0.1403 | 4.0 | 1096 | 0.5101 | 0.7481 | 0.8176 | 0.1156 | 0.8826 | 0.001 | | 0.1403 | 5.0 | 1370 | 0.5244 | 0.7614 | 0.8268 | 0.1136 | 0.8887 | 0.001 | | 0.1313 | 6.0 | 1644 | 0.5219 | 0.7509 | 0.8210 | 0.1110 | 0.8777 | 0.001 | | 0.1313 | 7.0 | 1918 | 0.5324 | 0.7614 | 0.8289 | 0.1085 | 0.8846 | 0.001 | | 0.1289 | 8.0 | 2192 | 0.5379 | 0.7711 | 0.8332 | 0.1101 | 0.8958 | 0.001 | | 0.1289 | 9.0 | 2466 | 0.5139 | 0.7670 | 0.8271 | 0.1113 | 0.8924 | 0.001 | | 0.1268 | 10.0 | 2740 | 0.5313 | 0.7611 | 0.8258 | 0.1138 | 0.8804 | 0.001 | | 0.1255 | 11.0 | 3014 | 0.5261 | 0.7627 | 0.8262 | 0.1139 | 0.8880 | 0.001 | | 0.1255 | 12.0 | 3288 | 0.5338 | 0.7573 | 0.8210 | 0.1121 | 0.8736 | 0.001 | | 0.1253 | 13.0 | 3562 | 0.5219 | 0.7489 | 0.8207 | 0.1111 | 0.8803 | 0.001 | | 0.1253 | 14.0 | 3836 | 0.5400 | 0.7777 | 0.8408 | 0.1025 | 0.8987 | 0.0001 | | 0.1171 | 15.0 | 4110 | 0.5404 | 0.7795 | 0.8429 | 0.0999 | 0.8973 | 0.0001 | | 0.1171 | 16.0 | 4384 | 0.5407 | 0.7861 | 0.8463 | 0.1008 | 0.9033 | 0.0001 | | 0.1107 | 17.0 | 4658 | 0.5459 | 0.7878 | 0.8474 | 0.1014 | 0.9055 | 0.0001 | | 0.1107 | 18.0 | 4932 | 0.5480 | 0.7868 | 0.8471 | 0.0973 | 0.9020 | 0.0001 | | 0.1078 | 19.0 | 5206 | 0.5480 | 0.7894 | 0.8491 | 0.0974 | 0.9054 | 0.0001 | | 0.1078 | 20.0 | 5480 | 0.5550 | 0.7948 | 0.8498 | 0.0971 | 0.9030 | 0.0001 | | 0.1061 | 21.0 | 5754 | 0.5532 | 0.7940 | 0.8509 | 0.0964 | 0.9081 | 0.0001 | | 0.1048 | 22.0 | 6028 | 0.5564 | 0.7974 | 0.8520 | 0.0962 | 0.9080 | 0.0001 | | 0.1048 | 23.0 | 6302 | 0.5585 | 0.7969 | 0.8505 | 0.0960 | 0.9012 | 0.0001 | | 0.1038 | 24.0 | 6576 | 0.5626 | 0.7974 | 0.8510 | 0.0951 | 0.9024 | 0.0001 | | 0.1038 | 25.0 | 6850 | 0.5644 | 0.7953 | 0.8512 | 0.0944 | 0.9012 | 0.0001 | | 0.1017 | 26.0 | 7124 | 0.5640 | 0.8037 | 0.8572 | 0.0948 | 0.9112 | 0.0001 | | 0.1017 | 27.0 | 7398 | 0.5637 | 0.8035 | 0.8551 | 0.0923 | 0.9086 | 0.0001 | | 0.1008 | 28.0 | 7672 | 0.5644 | 0.8073 | 0.8561 | 0.0919 | 0.9084 | 0.0001 | | 0.1008 | 29.0 | 7946 | 0.5682 | 0.8078 | 0.8572 | 0.0923 | 0.9082 | 0.0001 | | 0.1006 | 30.0 | 8220 | 0.5637 | 0.8079 | 0.8561 | 0.0924 | 0.9108 | 0.0001 | | 0.1006 | 31.0 | 8494 | 0.5689 | 0.8044 | 0.8549 | 0.0925 | 0.9050 | 0.0001 | | 0.0987 | 32.0 | 8768 | 0.5678 | 0.8071 | 0.8582 | 0.0913 | 0.9117 | 0.0001 | | 0.0983 | 33.0 | 9042 | 0.5692 | 0.8082 | 0.8571 | 0.0911 | 0.9061 | 0.0001 | | 0.0983 | 34.0 | 9316 | 0.5710 | 0.8060 | 0.8570 | 0.0906 | 0.9056 | 0.0001 | | 0.0967 | 35.0 | 9590 | 0.5692 | 0.8104 | 0.8578 | 0.0909 | 0.9083 | 0.0001 | | 0.0967 | 36.0 | 9864 | 0.5748 | 0.8114 | 0.8582 | 0.0917 | 0.9079 | 0.0001 | | 0.0963 | 37.0 | 10138 | 0.5741 | 0.8104 | 0.8572 | 0.0908 | 0.9057 | 0.0001 | | 0.0963 | 38.0 | 10412 | 0.5710 | 0.8136 | 0.8594 | 0.0910 | 0.9101 | 0.0001 | | 0.0957 | 39.0 | 10686 | 0.5685 | 0.8085 | 0.8577 | 0.0907 | 0.9098 | 0.0001 | | 0.0957 | 40.0 | 10960 | 0.5731 | 0.8112 | 0.8592 | 0.0903 | 0.9098 | 0.0001 | | 0.0953 | 41.0 | 11234 | 0.5717 | 0.8134 | 0.8586 | 0.0906 | 0.9087 | 0.0001 | | 0.0943 | 42.0 | 11508 | 0.5665 | 0.8136 | 0.8584 | 0.0903 | 0.9089 | 0.0001 | | 0.0943 | 43.0 | 11782 | 0.5699 | 0.8178 | 0.8604 | 0.0905 | 0.9132 | 0.0001 | | 0.0947 | 44.0 | 12056 | 0.5727 | 0.8149 | 0.8585 | 0.0910 | 0.9075 | 0.0001 | | 0.0947 | 45.0 | 12330 | 0.5727 | 0.8113 | 0.8591 | 0.0905 | 0.9081 | 0.0001 | | 0.0925 | 46.0 | 12604 | 0.5727 | 0.8139 | 0.8608 | 0.0896 | 0.9107 | 0.0001 | | 0.0925 | 47.0 | 12878 | 0.5745 | 0.8154 | 0.8599 | 0.0895 | 0.9079 | 0.0001 | | 0.0928 | 48.0 | 13152 | 0.5745 | 0.8155 | 0.8606 | 0.0896 | 0.9098 | 0.0001 | | 0.0928 | 49.0 | 13426 | 0.5727 | 0.8169 | 0.8606 | 0.0891 | 0.9131 | 0.0001 | | 0.0914 | 50.0 | 13700 | 0.5734 | 0.8183 | 0.8617 | 0.0895 | 0.9125 | 0.0001 | | 0.0914 | 51.0 | 13974 | 0.5668 | 0.8184 | 0.8608 | 0.0903 | 0.9149 | 0.0001 | | 0.0919 | 52.0 | 14248 | 0.5762 | 0.8172 | 0.8617 | 0.0904 | 0.9106 | 0.0001 | | 0.091 | 53.0 | 14522 | 0.5734 | 0.8154 | 0.8604 | 0.0911 | 0.9134 | 0.0001 | | 0.091 | 54.0 | 14796 | 0.5752 | 0.8224 | 0.8629 | 0.0909 | 0.9118 | 0.0001 | | 0.0907 | 55.0 | 15070 | 0.5720 | 0.8247 | 0.8628 | 0.0894 | 0.9151 | 0.0001 | | 0.0907 | 56.0 | 15344 | 0.5724 | 0.8197 | 0.8614 | 0.0895 | 0.9088 | 1e-05 | | 0.0883 | 57.0 | 15618 | 0.5755 | 0.8262 | 0.8653 | 0.0880 | 0.9160 | 1e-05 | | 0.0883 | 58.0 | 15892 | 0.5783 | 0.8227 | 0.8639 | 0.0885 | 0.9111 | 1e-05 | | 0.0872 | 59.0 | 16166 | 0.5765 | 0.8263 | 0.8655 | 0.0879 | 0.9161 | 1e-05 | | 0.0872 | 60.0 | 16440 | 0.5800 | 0.8238 | 0.8654 | 0.0884 | 0.9150 | 1e-05 | | 0.0873 | 61.0 | 16714 | 0.5745 | 0.8266 | 0.8652 | 0.0879 | 0.9168 | 1e-05 | | 0.0873 | 62.0 | 16988 | 0.5765 | 0.8252 | 0.8650 | 0.0880 | 0.9144 | 1e-05 | | 0.0864 | 63.0 | 17262 | 0.5800 | 0.8267 | 0.8650 | 0.0883 | 0.9134 | 1e-05 | | 0.086 | 64.0 | 17536 | 0.5783 | 0.8257 | 0.8667 | 0.0875 | 0.9178 | 1e-05 | | 0.086 | 65.0 | 17810 | 0.5811 | 0.8277 | 0.8670 | 0.0872 | 0.9159 | 1e-05 | | 0.0855 | 66.0 | 18084 | 0.5818 | 0.8263 | 0.8662 | 0.0873 | 0.9147 | 1e-05 | | 0.0855 | 67.0 | 18358 | 0.5797 | 0.8237 | 0.8648 | 0.0878 | 0.9121 | 1e-05 | | 0.0853 | 68.0 | 18632 | 0.5807 | 0.8233 | 0.8644 | 0.0879 | 0.9110 | 1e-05 | | 0.0853 | 69.0 | 18906 | 0.5832 | 0.8274 | 0.8654 | 0.0873 | 0.9129 | 1e-05 | | 0.0854 | 70.0 | 19180 | 0.5811 | 0.8287 | 0.8661 | 0.0873 | 0.9166 | 1e-05 | | 0.0854 | 71.0 | 19454 | 0.5779 | 0.8262 | 0.8657 | 0.0873 | 0.9156 | 1e-05 | | 0.0847 | 72.0 | 19728 | 0.5804 | 0.8279 | 0.8660 | 0.0873 | 0.9172 | 0.0000 | | 0.0852 | 73.0 | 20002 | 0.5765 | 0.8259 | 0.8662 | 0.0890 | 0.9175 | 0.0000 | | 0.0852 | 74.0 | 20276 | 0.5835 | 0.8267 | 0.8663 | 0.0871 | 0.9145 | 0.0000 | | 0.0845 | 75.0 | 20550 | 0.5762 | 0.8243 | 0.8651 | 0.0872 | 0.9151 | 0.0000 | | 0.0845 | 76.0 | 20824 | 0.5776 | 0.8258 | 0.8660 | 0.0871 | 0.9162 | 0.0000 | | 0.0849 | 77.0 | 21098 | 0.5779 | 0.8263 | 0.8655 | 0.0879 | 0.9152 | 0.0000 | | 0.0849 | 78.0 | 21372 | 0.5779 | 0.8241 | 0.8647 | 0.0883 | 0.9139 | 0.0000 | | 0.0853 | 79.0 | 21646 | 0.5807 | 0.8284 | 0.8667 | 0.0873 | 0.9170 | 0.0000 | | 0.0853 | 80.0 | 21920 | 0.5814 | 0.8258 | 0.8654 | 0.0873 | 0.9140 | 0.0000 | | 0.0838 | 81.0 | 22194 | 0.5828 | 0.8262 | 0.8654 | 0.0871 | 0.9132 | 0.0000 | | 0.0838 | 82.0 | 22468 | 0.5818 | 0.8253 | 0.8669 | 0.0874 | 0.9155 | 0.0000 | | 0.0842 | 83.0 | 22742 | 0.5846 | 0.8282 | 0.8667 | 0.0870 | 0.9161 | 0.0000 | | 0.0837 | 84.0 | 23016 | 0.0881 | 0.8627 | 0.8233 | 0.9080 | 0.5811 | 0.0000 | | 0.0837 | 85.0 | 23290 | 0.0871 | 0.8657 | 0.8277 | 0.9141 | 0.5807 | 0.0000 | ### Framework versions - Transformers 4.36.2 - Pytorch 2.1.0+cu118 - Datasets 2.14.5 - Tokenizers 0.15.0