dinov2-large-2024_01_24-with_data_aug_batch-size32_epochs93_freeze

DinoVd'eau is a fine-tuned version of facebook/dinov2-large on the multilabel_complete_dataset dataset. It achieves the following results on the evaluation set:

• Loss: 0.0864
• F1 Micro: 0.8668
• F1 Macro: 0.8381
• Roc Auc: 0.9138
• Accuracy: 0.5805

Model description

DinoVd'eau is a model built on top of dinov2 model for underwater multilabel image classification.The classification head is a combination of linear, ReLU, batch normalization, and dropout layers.

• Developed by: lombardata, credits to César Leblanc

Intended uses & limitations

You can use the raw model for classify diverse marine species, encompassing coral morphotypes classes taken from the Global Coral Reef Monitoring Network (GCRMN), habitats classes and seagrass species.

Training and evaluation data

Details on the number of images for each class are given in the following table:

	train	val	test	Total
Acropore_branched	804	202	200	1206
Acropore_digitised	465	108	101	674
Acropore_tabular	964	276	267	1507
Algae_assembly	2172	692	698	3562
Algae_limestone	1327	439	441	2207
Algae_sodding	2079	676	671	3426
Dead_coral	1126	358	355	1839
Fish	874	243	242	1359
Human_object	407	135	136	678
Living_coral	1765	580	571	2916
Millepore	350	119	102	571
No_acropore_encrusting	411	142	129	682
No_acropore_foliaceous	212	34	39	285
No_acropore_massive	921	317	310	1548
No_acropore_sub_massive	1205	362	363	1930
Rock	3736	1218	1217	6171
Sand	3594	1202	1194	5990
Scrap	2121	724	741	3586
Sea_cucumber	781	254	265	1300
Sea_urchins	189	60	72	321
Sponge	226	75	88	389
Syringodium_isoetifolium	1171	386	392	1949
Thalassodendron_ciliatum	783	261	260	1304
Useless	587	195	195	977

Training procedure

Data Augmentation

Data were augmented using the following transformations :

• training transformations : Sequential( (0): PreProcess() (1): Resize(output_size=(518, 518), p=1.0, p_batch=1.0, same_on_batch=True, size=(518, 518), side=short, resample=bilinear, align_corners=True, antialias=False) (2): RandomHorizontalFlip(p=0.25, p_batch=1.0, same_on_batch=False) (3): RandomVerticalFlip(p=0.25, p_batch=1.0, same_on_batch=False) (4): ColorJiggle(brightness=0.0, contrast=0.0, saturation=0.0, hue=0.0, p=0.25, p_batch=1.0, same_on_batch=False) (5): RandomPerspective(distortion_scale=0.5, p=0.25, p_batch=1.0, same_on_batch=False, align_corners=False, resample=bilinear) (6): Normalize(p=1.0, p_batch=1.0, same_on_batch=True, mean=tensor([0.4850, 0.4560, 0.4060]), std=tensor([0.2290, 0.2240, 0.2250])) )
• validation transformations : Sequential( (0): PreProcess() (1): Resize(output_size=(518, 518), p=1.0, p_batch=1.0, same_on_batch=True, size=(518, 518), side=short, resample=bilinear, align_corners=True, antialias=False) (2): Normalize(p=1.0, p_batch=1.0, same_on_batch=True, mean=tensor([0.4850, 0.4560, 0.4060]), std=tensor([0.2290, 0.2240, 0.2250])) )

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: ReduceLROnPlateau with a patience of 5 epochs and a factor of 0.1
• freeze_encoder: True
• num_epochs: 110

Training results

Training Loss	Epoch	Step	Accuracy	F1 Macro	F1 Micro	Validation Loss	Roc Auc	Rate
No log	1.0	274	0.4589	0.6395	0.7738	0.1359	0.8471	0.001
0.2459	2.0	548	0.4941	0.7305	0.8032	0.1236	0.8697	0.001
0.2459	3.0	822	0.5125	0.7426	0.8174	0.1167	0.8828	0.001
0.1403	4.0	1096	0.5101	0.7481	0.8176	0.1156	0.8826	0.001
0.1403	5.0	1370	0.5244	0.7614	0.8268	0.1136	0.8887	0.001
0.1313	6.0	1644	0.5219	0.7509	0.8210	0.1110	0.8777	0.001
0.1313	7.0	1918	0.5324	0.7614	0.8289	0.1085	0.8846	0.001
0.1289	8.0	2192	0.5379	0.7711	0.8332	0.1101	0.8958	0.001
0.1289	9.0	2466	0.5139	0.7670	0.8271	0.1113	0.8924	0.001
0.1268	10.0	2740	0.5313	0.7611	0.8258	0.1138	0.8804	0.001
0.1255	11.0	3014	0.5261	0.7627	0.8262	0.1139	0.8880	0.001
0.1255	12.0	3288	0.5338	0.7573	0.8210	0.1121	0.8736	0.001
0.1253	13.0	3562	0.5219	0.7489	0.8207	0.1111	0.8803	0.001
0.1253	14.0	3836	0.5400	0.7777	0.8408	0.1025	0.8987	0.0001
0.1171	15.0	4110	0.5404	0.7795	0.8429	0.0999	0.8973	0.0001
0.1171	16.0	4384	0.5407	0.7861	0.8463	0.1008	0.9033	0.0001
0.1107	17.0	4658	0.5459	0.7878	0.8474	0.1014	0.9055	0.0001
0.1107	18.0	4932	0.5480	0.7868	0.8471	0.0973	0.9020	0.0001
0.1078	19.0	5206	0.5480	0.7894	0.8491	0.0974	0.9054	0.0001
0.1078	20.0	5480	0.5550	0.7948	0.8498	0.0971	0.9030	0.0001
0.1061	21.0	5754	0.5532	0.7940	0.8509	0.0964	0.9081	0.0001
0.1048	22.0	6028	0.5564	0.7974	0.8520	0.0962	0.9080	0.0001
0.1048	23.0	6302	0.5585	0.7969	0.8505	0.0960	0.9012	0.0001
0.1038	24.0	6576	0.5626	0.7974	0.8510	0.0951	0.9024	0.0001
0.1038	25.0	6850	0.5644	0.7953	0.8512	0.0944	0.9012	0.0001
0.1017	26.0	7124	0.5640	0.8037	0.8572	0.0948	0.9112	0.0001
0.1017	27.0	7398	0.5637	0.8035	0.8551	0.0923	0.9086	0.0001
0.1008	28.0	7672	0.5644	0.8073	0.8561	0.0919	0.9084	0.0001
0.1008	29.0	7946	0.5682	0.8078	0.8572	0.0923	0.9082	0.0001
0.1006	30.0	8220	0.5637	0.8079	0.8561	0.0924	0.9108	0.0001
0.1006	31.0	8494	0.5689	0.8044	0.8549	0.0925	0.9050	0.0001
0.0987	32.0	8768	0.5678	0.8071	0.8582	0.0913	0.9117	0.0001
0.0983	33.0	9042	0.5692	0.8082	0.8571	0.0911	0.9061	0.0001
0.0983	34.0	9316	0.5710	0.8060	0.8570	0.0906	0.9056	0.0001
0.0967	35.0	9590	0.5692	0.8104	0.8578	0.0909	0.9083	0.0001
0.0967	36.0	9864	0.5748	0.8114	0.8582	0.0917	0.9079	0.0001
0.0963	37.0	10138	0.5741	0.8104	0.8572	0.0908	0.9057	0.0001
0.0963	38.0	10412	0.5710	0.8136	0.8594	0.0910	0.9101	0.0001
0.0957	39.0	10686	0.5685	0.8085	0.8577	0.0907	0.9098	0.0001
0.0957	40.0	10960	0.5731	0.8112	0.8592	0.0903	0.9098	0.0001
0.0953	41.0	11234	0.5717	0.8134	0.8586	0.0906	0.9087	0.0001
0.0943	42.0	11508	0.5665	0.8136	0.8584	0.0903	0.9089	0.0001
0.0943	43.0	11782	0.5699	0.8178	0.8604	0.0905	0.9132	0.0001
0.0947	44.0	12056	0.5727	0.8149	0.8585	0.0910	0.9075	0.0001
0.0947	45.0	12330	0.5727	0.8113	0.8591	0.0905	0.9081	0.0001
0.0925	46.0	12604	0.5727	0.8139	0.8608	0.0896	0.9107	0.0001
0.0925	47.0	12878	0.5745	0.8154	0.8599	0.0895	0.9079	0.0001
0.0928	48.0	13152	0.5745	0.8155	0.8606	0.0896	0.9098	0.0001
0.0928	49.0	13426	0.5727	0.8169	0.8606	0.0891	0.9131	0.0001
0.0914	50.0	13700	0.5734	0.8183	0.8617	0.0895	0.9125	0.0001
0.0914	51.0	13974	0.5668	0.8184	0.8608	0.0903	0.9149	0.0001
0.0919	52.0	14248	0.5762	0.8172	0.8617	0.0904	0.9106	0.0001
0.091	53.0	14522	0.5734	0.8154	0.8604	0.0911	0.9134	0.0001
0.091	54.0	14796	0.5752	0.8224	0.8629	0.0909	0.9118	0.0001
0.0907	55.0	15070	0.5720	0.8247	0.8628	0.0894	0.9151	0.0001
0.0907	56.0	15344	0.5724	0.8197	0.8614	0.0895	0.9088	1e-05
0.0883	57.0	15618	0.5755	0.8262	0.8653	0.0880	0.9160	1e-05
0.0883	58.0	15892	0.5783	0.8227	0.8639	0.0885	0.9111	1e-05
0.0872	59.0	16166	0.5765	0.8263	0.8655	0.0879	0.9161	1e-05
0.0872	60.0	16440	0.5800	0.8238	0.8654	0.0884	0.9150	1e-05
0.0873	61.0	16714	0.5745	0.8266	0.8652	0.0879	0.9168	1e-05
0.0873	62.0	16988	0.5765	0.8252	0.8650	0.0880	0.9144	1e-05
0.0864	63.0	17262	0.5800	0.8267	0.8650	0.0883	0.9134	1e-05
0.086	64.0	17536	0.5783	0.8257	0.8667	0.0875	0.9178	1e-05
0.086	65.0	17810	0.5811	0.8277	0.8670	0.0872	0.9159	1e-05
0.0855	66.0	18084	0.5818	0.8263	0.8662	0.0873	0.9147	1e-05
0.0855	67.0	18358	0.5797	0.8237	0.8648	0.0878	0.9121	1e-05
0.0853	68.0	18632	0.5807	0.8233	0.8644	0.0879	0.9110	1e-05
0.0853	69.0	18906	0.5832	0.8274	0.8654	0.0873	0.9129	1e-05
0.0854	70.0	19180	0.5811	0.8287	0.8661	0.0873	0.9166	1e-05
0.0854	71.0	19454	0.5779	0.8262	0.8657	0.0873	0.9156	1e-05
0.0847	72.0	19728	0.5804	0.8279	0.8660	0.0873	0.9172	0.0000
0.0852	73.0	20002	0.5765	0.8259	0.8662	0.0890	0.9175	0.0000
0.0852	74.0	20276	0.5835	0.8267	0.8663	0.0871	0.9145	0.0000
0.0845	75.0	20550	0.5762	0.8243	0.8651	0.0872	0.9151	0.0000
0.0845	76.0	20824	0.5776	0.8258	0.8660	0.0871	0.9162	0.0000
0.0849	77.0	21098	0.5779	0.8263	0.8655	0.0879	0.9152	0.0000
0.0849	78.0	21372	0.5779	0.8241	0.8647	0.0883	0.9139	0.0000
0.0853	79.0	21646	0.5807	0.8284	0.8667	0.0873	0.9170	0.0000
0.0853	80.0	21920	0.5814	0.8258	0.8654	0.0873	0.9140	0.0000
0.0838	81.0	22194	0.5828	0.8262	0.8654	0.0871	0.9132	0.0000
0.0838	82.0	22468	0.5818	0.8253	0.8669	0.0874	0.9155	0.0000
0.0842	83.0	22742	0.5846	0.8282	0.8667	0.0870	0.9161	0.0000
0.0837	84.0	23016	0.0881	0.8627	0.8233	0.9080	0.5811	0.0000
0.0837	85.0	23290	0.0871	0.8657	0.8277	0.9141	0.5807	0.0000
0.0852	86.0	23564	0.0873	0.8655	0.8265	0.9144	0.5835	0.0000
0.0852	87.0	23838	0.0873	0.8649	0.8263	0.9136	0.5790	0.0000
0.084	88.0	24112	0.0870	0.8668	0.8268	0.9171	0.5800	0.0000
0.084	89.0	24386	0.0874	0.8657	0.8259	0.9134	0.5814	0.0000
0.0852	90.0	24660	0.0872	0.8672	0.8300	0.9160	0.5821	0.0000
0.0852	91.0	24934	0.0871	0.8663	0.8264	0.9141	0.5846	0.0000
0.0839	92.0	25208	0.0878	0.8645	0.8248	0.9114	0.5818	0.0000
0.0839	93.0	25482	0.0880	0.8666	0.8273	0.9163	0.5786	0.0000

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.15.0