DinoVd'eau is a fine-tuned version of facebook/dinov2-large. It achieves the following results on the test set:
- Loss: 0.1211
- F1 Micro: 0.8216
- F1 Macro: 0.7251
- Roc Auc: 0.8809
- Accuracy: 0.3068
Model description
DinoVd'eau is a model built on top of dinov2 model for underwater multilabel image classification.The classification head is a combination of linear, ReLU, batch normalization, and dropout layers.
The source code for training the model can be found in this Git repository.
- Developed by: lombardata, credits to César Leblanc and Victor Illien
Intended uses & limitations
You can use the raw model for classify diverse marine species, encompassing coral morphotypes classes taken from the Global Coral Reef Monitoring Network (GCRMN), habitats classes and seagrass species.
Training and evaluation data
Details on the number of images for each class are given in the following table:
Class | train | val | test | Total |
---|---|---|---|---|
Acropore_branched | 1472 | 477 | 459 | 2408 |
Acropore_digitised | 565 | 161 | 162 | 888 |
Acropore_sub_massive | 150 | 47 | 46 | 243 |
Acropore_tabular | 997 | 297 | 295 | 1589 |
Algae_assembly | 2545 | 836 | 857 | 4238 |
Algae_drawn_up | 366 | 129 | 125 | 620 |
Algae_limestone | 1650 | 563 | 559 | 2772 |
Algae_sodding | 3147 | 991 | 979 | 5117 |
Atra/Leucospilota | 1084 | 345 | 363 | 1792 |
Bleached_coral | 219 | 71 | 70 | 360 |
Blurred | 190 | 62 | 68 | 320 |
Dead_coral | 1979 | 642 | 643 | 3264 |
Fish | 2022 | 658 | 641 | 3321 |
Homo_sapiens | 161 | 59 | 62 | 282 |
Human_object | 156 | 56 | 58 | 270 |
Living_coral | 400 | 148 | 153 | 701 |
Millepore | 386 | 128 | 123 | 637 |
No_acropore_encrusting | 440 | 142 | 143 | 725 |
No_acropore_foliaceous | 203 | 39 | 44 | 286 |
No_acropore_massive | 1030 | 344 | 331 | 1705 |
No_acropore_solitary | 202 | 55 | 46 | 303 |
No_acropore_sub_massive | 1402 | 431 | 423 | 2256 |
Rock | 4483 | 1489 | 1485 | 7457 |
Rubble | 3093 | 1006 | 1024 | 5123 |
Sand | 5839 | 1942 | 1938 | 9719 |
Sea_cucumber | 1411 | 433 | 450 | 2294 |
Sea_urchins | 328 | 108 | 109 | 545 |
Sponge | 270 | 104 | 96 | 470 |
Syringodium_isoetifolium | 1213 | 393 | 389 | 1995 |
Thalassodendron_ciliatum | 781 | 261 | 261 | 1303 |
Useless | 579 | 193 | 193 | 965 |
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- Number of Epochs: 150
- Learning Rate: 0.001
- Train Batch Size: 32
- Eval Batch Size: 32
- Optimizer: Adam
- LR Scheduler Type: ReduceLROnPlateau with a patience of 5 epochs and a factor of 0.1
- Freeze Encoder: Yes
- Data Augmentation: Yes
Data Augmentation
Data were augmented using the following transformations :
Train Transforms
- PreProcess: No additional parameters
- Resize: probability=1.00
- RandomHorizontalFlip: probability=0.25
- RandomVerticalFlip: probability=0.25
- ColorJiggle: probability=0.25
- RandomPerspective: probability=0.25
- Normalize: probability=1.00
Val Transforms
- PreProcess: No additional parameters
- Resize: probability=1.00
- Normalize: probability=1.00
Training results
Epoch | Validation Loss | Accuracy | F1 Macro | F1 Micro | Learning Rate |
---|---|---|---|---|---|
1 | 0.1696680188179016 | 0.23079584775086506 | 0.7387644202930826 | 0.48240224551051036 | 0.001 |
2 | 0.15703289210796356 | 0.2342560553633218 | 0.7582120763265662 | 0.5678522534916328 | 0.001 |
3 | 0.15008607506752014 | 0.26089965397923875 | 0.7749956978144897 | 0.6070786785177279 | 0.001 |
4 | 0.1459328830242157 | 0.26332179930795846 | 0.7809972180612026 | 0.618893775222464 | 0.001 |
5 | 0.14656734466552734 | 0.257439446366782 | 0.7809636417381022 | 0.6145141477329354 | 0.001 |
6 | 0.14571721851825714 | 0.2726643598615917 | 0.7776683649441282 | 0.6059053020982528 | 0.001 |
7 | 0.14370936155319214 | 0.2577854671280277 | 0.7765906414896865 | 0.6237167782687788 | 0.001 |
8 | 0.14394836127758026 | 0.2685121107266436 | 0.7783921362120415 | 0.6388647657762934 | 0.001 |
9 | 0.14525611698627472 | 0.255363321799308 | 0.7855017808506181 | 0.6320393416159836 | 0.001 |
10 | 0.1438504010438919 | 0.27058823529411763 | 0.7851148597100958 | 0.6486588102502582 | 0.001 |
11 | 0.14624632894992828 | 0.2629757785467128 | 0.7877580433216058 | 0.6417150090850962 | 0.001 |
12 | 0.14636772871017456 | 0.25432525951557095 | 0.7837535496084674 | 0.639315846277914 | 0.001 |
13 | 0.14202021062374115 | 0.2754325259515571 | 0.78815800699599 | 0.6473402316193251 | 0.001 |
14 | 0.1542871743440628 | 0.25882352941176473 | 0.7832956146718476 | 0.6229110923470623 | 0.001 |
15 | 0.14085648953914642 | 0.2740484429065744 | 0.7903608334745046 | 0.6518642313031997 | 0.001 |
16 | 0.14292311668395996 | 0.27439446366782005 | 0.7863712916180852 | 0.6365564755176403 | 0.001 |
17 | 0.13990001380443573 | 0.2799307958477509 | 0.7929570954321973 | 0.6487058751449585 | 0.001 |
18 | 0.13916312158107758 | 0.2643598615916955 | 0.7884885951822639 | 0.6417719350399551 | 0.001 |
19 | 0.13893043994903564 | 0.26505190311418686 | 0.7925466885881707 | 0.655317550465269 | 0.001 |
20 | 0.14047470688819885 | 0.2778546712802768 | 0.7910536610255974 | 0.6519764438062028 | 0.001 |
21 | 0.13958919048309326 | 0.28096885813148786 | 0.7944965603502189 | 0.6586810047475636 | 0.001 |
22 | 0.1409333050251007 | 0.27231833910034603 | 0.788903634573398 | 0.6490927552938025 | 0.001 |
23 | 0.14421699941158295 | 0.271280276816609 | 0.791997969886652 | 0.6546007121389574 | 0.001 |
24 | 0.13853740692138672 | 0.28512110726643597 | 0.7920758550626482 | 0.6473707882350964 | 0.001 |
25 | 0.1393292397260666 | 0.28269896193771626 | 0.7918592325630697 | 0.6582355138095511 | 0.001 |
26 | 0.1404317170381546 | 0.2771626297577855 | 0.7923397169025812 | 0.6563032405626342 | 0.001 |
27 | 0.13901519775390625 | 0.27854671280276816 | 0.7912935429967007 | 0.6372533033632628 | 0.001 |
28 | 0.13772021234035492 | 0.2709342560553633 | 0.7872712947561937 | 0.6456065158774918 | 0.001 |
29 | 0.13838444650173187 | 0.2771626297577855 | 0.7939749377663389 | 0.6443851050256405 | 0.001 |
30 | 0.13830840587615967 | 0.2640138408304498 | 0.7942442400202548 | 0.6491486219528562 | 0.001 |
31 | 0.14057572185993195 | 0.27612456747404845 | 0.7925935235222654 | 0.6310327603311888 | 0.001 |
32 | 0.1388920247554779 | 0.2833910034602076 | 0.7971736261865237 | 0.6698724179590195 | 0.001 |
33 | 0.1613842248916626 | 0.27024221453287195 | 0.7971170574103223 | 0.6497178610524192 | 0.001 |
34 | 0.14063873887062073 | 0.284083044982699 | 0.7956378222537918 | 0.6584094608801518 | 0.001 |
35 | 0.1311163306236267 | 0.2903114186851211 | 0.8038767563907229 | 0.6783097245999954 | 0.0001 |
36 | 0.13003353774547577 | 0.2996539792387543 | 0.8076212858821555 | 0.6793622509723133 | 0.0001 |
37 | 0.1292405128479004 | 0.3013840830449827 | 0.8050973873758684 | 0.6802670830914382 | 0.0001 |
38 | 0.1283079981803894 | 0.3017301038062284 | 0.8097280639653521 | 0.691530412631533 | 0.0001 |
39 | 0.12849266827106476 | 0.30207612456747407 | 0.8103419376466646 | 0.6858958437183065 | 0.0001 |
40 | 0.127731055021286 | 0.30103806228373703 | 0.8110104061765693 | 0.6878904051166272 | 0.0001 |
41 | 0.12703965604305267 | 0.3079584775086505 | 0.8139767888452869 | 0.6967877061883809 | 0.0001 |
42 | 0.12826678156852722 | 0.30311418685121105 | 0.8101190476190476 | 0.6900880377263761 | 0.0001 |
43 | 0.126459002494812 | 0.3103806228373702 | 0.813363476513439 | 0.698512042551847 | 0.0001 |
44 | 0.12749163806438446 | 0.3110726643598616 | 0.8125052384544464 | 0.6928688967130782 | 0.0001 |
45 | 0.1258065104484558 | 0.3110726643598616 | 0.8120326165025983 | 0.6905501287813828 | 0.0001 |
46 | 0.12585382163524628 | 0.30934256055363324 | 0.8141467874657136 | 0.7029042057423242 | 0.0001 |
47 | 0.12546662986278534 | 0.3096885813148789 | 0.8159557335755873 | 0.7047603619742887 | 0.0001 |
48 | 0.1253565400838852 | 0.3058823529411765 | 0.8139360013367867 | 0.6974661734658276 | 0.0001 |
49 | 0.12485454976558685 | 0.31141868512110726 | 0.8143054798540942 | 0.6943635260846596 | 0.0001 |
50 | 0.12531296908855438 | 0.3103806228373702 | 0.8110079294406988 | 0.6932075391775876 | 0.0001 |
51 | 0.12453079223632812 | 0.31384083044982697 | 0.814743856726364 | 0.6991762751520911 | 0.0001 |
52 | 0.12411220371723175 | 0.31557093425605537 | 0.8162597533350113 | 0.7046749806541895 | 0.0001 |
53 | 0.12500016391277313 | 0.30899653979238756 | 0.8145090681676048 | 0.6997083670176113 | 0.0001 |
54 | 0.12449914962053299 | 0.31453287197231833 | 0.8163877092180865 | 0.704905692622788 | 0.0001 |
55 | 0.1243244931101799 | 0.3134948096885813 | 0.8149173588388289 | 0.7015504397667252 | 0.0001 |
56 | 0.12355821579694748 | 0.3107266435986159 | 0.8157839909996251 | 0.7057673948207344 | 0.0001 |
57 | 0.12367301434278488 | 0.3134948096885813 | 0.8154531696653775 | 0.7048308334487045 | 0.0001 |
58 | 0.12418670952320099 | 0.31418685121107265 | 0.8152379352345752 | 0.702233816965674 | 0.0001 |
59 | 0.12396683543920517 | 0.3169550173010381 | 0.818200535309862 | 0.7033190719353933 | 0.0001 |
60 | 0.12383627891540527 | 0.31591695501730105 | 0.8169142667441281 | 0.707100255674436 | 0.0001 |
61 | 0.12376978993415833 | 0.3134948096885813 | 0.8161918288913026 | 0.7057906494932572 | 0.0001 |
62 | 0.12311123311519623 | 0.3176470588235294 | 0.8186829126776908 | 0.710084215624648 | 0.0001 |
63 | 0.12293447554111481 | 0.3169550173010381 | 0.8167094358672332 | 0.7034728877779122 | 0.0001 |
64 | 0.12318814545869827 | 0.314878892733564 | 0.8167272423444876 | 0.7039496449567197 | 0.0001 |
65 | 0.12292256951332092 | 0.314878892733564 | 0.8155872667398464 | 0.7017054590441031 | 0.0001 |
66 | 0.1233849748969078 | 0.31626297577854673 | 0.8159070972053971 | 0.7041658521763916 | 0.0001 |
67 | 0.12293217331171036 | 0.31626297577854673 | 0.8193559069459393 | 0.7115649077332132 | 0.0001 |
68 | 0.12244237214326859 | 0.32041522491349483 | 0.8191163512776403 | 0.7117842251236416 | 0.0001 |
69 | 0.12294486910104752 | 0.3169550173010381 | 0.8166987367188154 | 0.7100473926398085 | 0.0001 |
70 | 0.1227104440331459 | 0.3176470588235294 | 0.8194294269317142 | 0.7073677647969174 | 0.0001 |
71 | 0.12281835079193115 | 0.3221453287197232 | 0.8190611783426155 | 0.7116912727578291 | 0.0001 |
72 | 0.12274286150932312 | 0.3190311418685121 | 0.8180748213535098 | 0.7076888299444815 | 0.0001 |
73 | 0.12224046140909195 | 0.3197231833910035 | 0.8208955223880596 | 0.7115787794437213 | 0.0001 |
74 | 0.12199588865041733 | 0.3197231833910035 | 0.8202228504906037 | 0.7118683173431358 | 0.0001 |
75 | 0.12203484028577805 | 0.3152249134948097 | 0.8189331329827199 | 0.712534792002447 | 0.0001 |
76 | 0.12196851521730423 | 0.31626297577854673 | 0.8175121379541269 | 0.7098431444159636 | 0.0001 |
77 | 0.12242470681667328 | 0.31418685121107265 | 0.8165241925295827 | 0.7095722503090334 | 0.0001 |
78 | 0.12182802706956863 | 0.3197231833910035 | 0.8182161594963274 | 0.7104246992338475 | 0.0001 |
79 | 0.12361899018287659 | 0.31868512110726643 | 0.819422117802234 | 0.7041915269998454 | 0.0001 |
80 | 0.12182050198316574 | 0.32110726643598614 | 0.8200709161375442 | 0.7215166575729017 | 0.0001 |
81 | 0.1222558245062828 | 0.3190311418685121 | 0.8196857594147477 | 0.7142296348553671 | 0.0001 |
82 | 0.12162397801876068 | 0.3193771626297578 | 0.8217637573903337 | 0.7142487132976763 | 0.0001 |
83 | 0.12116336822509766 | 0.31591695501730105 | 0.8199182584035366 | 0.7157416946786116 | 0.0001 |
84 | 0.12182576209306717 | 0.31591695501730105 | 0.8160822656776805 | 0.7005118683471125 | 0.0001 |
85 | 0.12157247215509415 | 0.32006920415224915 | 0.8195705855271419 | 0.7112117400824071 | 0.0001 |
86 | 0.12132923305034637 | 0.32076124567474046 | 0.8199150778453084 | 0.7130384438706758 | 0.0001 |
87 | 0.1209079846739769 | 0.3245674740484429 | 0.8217034910331626 | 0.7196634291835953 | 0.0001 |
88 | 0.12118621915578842 | 0.32110726643598614 | 0.8202653799758746 | 0.7133004794787455 | 0.0001 |
89 | 0.12148680537939072 | 0.3197231833910035 | 0.8190659042530991 | 0.7130024863292826 | 0.0001 |
90 | 0.12146373093128204 | 0.32110726643598614 | 0.8195863233431828 | 0.7138227751305745 | 0.0001 |
91 | 0.12076817452907562 | 0.3245674740484429 | 0.8219622861424468 | 0.7201706813878868 | 0.0001 |
92 | 0.12100402265787125 | 0.3190311418685121 | 0.8212632275132274 | 0.7167137675234206 | 0.0001 |
93 | 0.12075362354516983 | 0.3169550173010381 | 0.8210482355397788 | 0.7168209309560042 | 0.0001 |
94 | 0.12131747603416443 | 0.3169550173010381 | 0.8207492316637593 | 0.7129564562249288 | 0.0001 |
95 | 0.12093706429004669 | 0.31626297577854673 | 0.8198186907298326 | 0.7107923514896998 | 0.0001 |
96 | 0.12080572545528412 | 0.31833910034602075 | 0.8205149675528575 | 0.7142082592350744 | 0.0001 |
97 | 0.12148387730121613 | 0.3221453287197232 | 0.821143001430856 | 0.7178627573063694 | 0.0001 |
98 | 0.12116367369890213 | 0.31833910034602075 | 0.8204551137784446 | 0.7220163155986198 | 0.0001 |
99 | 0.122224360704422 | 0.3190311418685121 | 0.8202876735240007 | 0.7148002470224453 | 0.0001 |
100 | 0.12044612318277359 | 0.3221453287197232 | 0.8234614273331406 | 0.7237819678241929 | 1e-05 |
101 | 0.12046819180250168 | 0.3214532871972318 | 0.8225361531388448 | 0.7175715394879117 | 1e-05 |
102 | 0.11973976343870163 | 0.3217993079584775 | 0.823289155622239 | 0.725621847387927 | 1e-05 |
103 | 0.12033144384622574 | 0.3214532871972318 | 0.8229430576899115 | 0.7238847645251945 | 1e-05 |
104 | 0.12013406306505203 | 0.32110726643598614 | 0.8229179659889403 | 0.7190340295360067 | 1e-05 |
105 | 0.12038043141365051 | 0.3190311418685121 | 0.8231939163498099 | 0.7188152152796179 | 1e-05 |
106 | 0.12012088298797607 | 0.3217993079584775 | 0.8225415657114211 | 0.7162169221873866 | 1e-05 |
107 | 0.11983397603034973 | 0.3214532871972318 | 0.8245079469653298 | 0.7198857151250498 | 1e-05 |
108 | 0.11995845288038254 | 0.32041522491349483 | 0.8219728688661866 | 0.7150987178757772 | 1e-05 |
109 | 0.11981397867202759 | 0.3262975778546713 | 0.8223654125961338 | 0.719215254545586 | 1.0000000000000002e-06 |
110 | 0.11961892992258072 | 0.3238754325259516 | 0.8238556410893139 | 0.7210699392652612 | 1.0000000000000002e-06 |
111 | 0.11969973891973495 | 0.3256055363321799 | 0.8240756093516829 | 0.7220636143470806 | 1.0000000000000002e-06 |
112 | 0.11957906186580658 | 0.3235294117647059 | 0.8237543115987199 | 0.7203145097521634 | 1.0000000000000002e-06 |
113 | 0.11939564347267151 | 0.32283737024221454 | 0.8233045212765958 | 0.7178685303113534 | 1.0000000000000002e-06 |
114 | 0.12025978416204453 | 0.32006920415224915 | 0.8221373411940546 | 0.7198020940141643 | 1.0000000000000002e-06 |
115 | 0.12016712129116058 | 0.32283737024221454 | 0.8237909034780819 | 0.7216303606521312 | 1.0000000000000002e-06 |
116 | 0.12020432949066162 | 0.32491349480968856 | 0.8244099021301093 | 0.724344791703263 | 1.0000000000000002e-06 |
117 | 0.11986654251813889 | 0.3231833910034602 | 0.8214361006872708 | 0.7166667142633425 | 1.0000000000000002e-06 |
118 | 0.11982504278421402 | 0.31868512110726643 | 0.8216139293487359 | 0.7175073424210379 | 1.0000000000000002e-06 |
119 | 0.11932501941919327 | 0.3231833910034602 | 0.8241461080833714 | 0.724230778846038 | 1.0000000000000002e-06 |
120 | 0.11970102787017822 | 0.32283737024221454 | 0.8241621845262722 | 0.7204544328453095 | 1.0000000000000002e-06 |
121 | 0.11957091093063354 | 0.3221453287197232 | 0.8231556948798329 | 0.7230743210923909 | 1.0000000000000002e-06 |
122 | 0.1201123297214508 | 0.32041522491349483 | 0.8219534727471236 | 0.7196142292960622 | 1.0000000000000002e-06 |
123 | 0.11973411589860916 | 0.3217993079584775 | 0.8251494537208823 | 0.7239831576514587 | 1.0000000000000002e-06 |
124 | 0.11995533108711243 | 0.3221453287197232 | 0.8215739769150053 | 0.7195780687908326 | 1.0000000000000002e-06 |
125 | 0.11960428953170776 | 0.31868512110726643 | 0.8227369121377187 | 0.7221896000631038 | 1.0000000000000002e-06 |
126 | 0.11972622573375702 | 0.3235294117647059 | 0.8236807168934618 | 0.7213769755798427 | 1.0000000000000002e-07 |
127 | 0.11995424330234528 | 0.3197231833910035 | 0.8234563813938091 | 0.7220369847258239 | 1.0000000000000002e-07 |
128 | 0.11965782195329666 | 0.3245674740484429 | 0.823190583521162 | 0.7168496424502305 | 1.0000000000000002e-07 |
129 | 0.11992616206407547 | 0.3256055363321799 | 0.8250164690382081 | 0.7255628468971802 | 1.0000000000000002e-07 |
130 | 0.11971699446439743 | 0.3217993079584775 | 0.8222776392352452 | 0.7171476023644834 | 1.0000000000000002e-07 |
131 | 0.1200244128704071 | 0.32491349480968856 | 0.8241812768772743 | 0.7202861226868075 | 1.0000000000000002e-07 |
132 | 0.11962693929672241 | 0.3238754325259516 | 0.8228243226370333 | 0.7204093851990869 | 1.0000000000000004e-08 |
CO2 Emissions
The estimated CO2 emissions for training this model are documented below:
- Emissions: 0.159156917762164 grams of CO2
- Source: Code Carbon
- Training Type: fine-tuning
- Geographical Location: Brest, France
- Hardware Used: NVIDIA Tesla V100 PCIe 32 Go
Framework Versions
- Transformers: 4.41.1
- Pytorch: 2.3.0+cu121
- Datasets: 2.19.1
- Tokenizers: 0.19.1
- Downloads last month
- 16