dino-base-2023_12_01-with_custom_small_head

This model is a fine-tuned version of facebook/dinov2-base on the multilabel_complete_dataset dataset. It achieves the following results on the evaluation set:

• Loss: 0.1266
• F1 Micro: 0.8318
• F1 Macro: 0.8018
• Roc Auc: 0.8960
• Accuracy: 0.5224
• Learning Rate: 0.0000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.01
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 90

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Micro	F1 Macro	Roc Auc	Accuracy	Rate
0.4702	1.0	536	0.4545	0.7549	0.6933	0.8356	0.4298	0.01
0.4091	2.0	1072	0.3691	0.7832	0.7315	0.8762	0.4377	0.01
0.3999	3.0	1608	0.4631	0.7725	0.7214	0.8643	0.4141	0.01
0.3958	4.0	2144	0.4028	0.7858	0.7310	0.8765	0.4534	0.01
0.3795	5.0	2680	0.5129	0.7452	0.6974	0.8459	0.4230	0.01
0.3877	6.0	3216	0.5185	0.7386	0.6805	0.8180	0.4259	0.01
0.3658	7.0	3752	0.4688	0.7710	0.7034	0.8525	0.4391	0.01
0.373	8.0	4288	0.5070	0.7608	0.7020	0.8647	0.3859	0.01
0.2911	9.0	4824	0.2327	0.8213	0.7938	0.8885	0.5088	0.001
0.139	10.0	5360	0.2238	0.8193	0.7891	0.8972	0.4987	0.001
0.1187	11.0	5896	0.2095	0.8169	0.7858	0.8833	0.5084	0.001
0.1084	12.0	6432	0.1985	0.8209	0.7995	0.9031	0.4959	0.001
0.1038	13.0	6968	0.1949	0.8186	0.7914	0.8941	0.4902	0.001
0.0936	14.0	7504	0.1806	0.8243	0.7855	0.8947	0.5013	0.001
0.0915	15.0	8040	0.1799	0.8163	0.7788	0.8806	0.5113	0.001
0.0875	16.0	8576	0.1739	0.8196	0.7848	0.8894	0.5027	0.001
0.0848	17.0	9112	0.1719	0.8224	0.7894	0.9007	0.4898	0.001
0.0861	18.0	9648	0.1686	0.8212	0.7839	0.8904	0.4962	0.001
0.0845	19.0	10184	0.1659	0.8179	0.7830	0.8934	0.4941	0.001
0.0824	20.0	10720	0.1743	0.8105	0.7826	0.8839	0.4930	0.001
0.0834	21.0	11256	0.1601	0.8183	0.7905	0.8959	0.4977	0.001
0.0803	22.0	11792	0.1617	0.8206	0.7885	0.8985	0.4970	0.001
0.0817	23.0	12328	0.1586	0.8190	0.7893	0.8900	0.5038	0.001
0.0821	24.0	12864	0.1561	0.8203	0.7798	0.8825	0.5148	0.001
0.0795	25.0	13400	0.1552	0.8208	0.7897	0.8981	0.5013	0.001
0.0792	26.0	13936	0.1544	0.8165	0.7844	0.8853	0.5048	0.001
0.0799	27.0	14472	0.1509	0.8235	0.7845	0.8889	0.5105	0.001
0.0795	28.0	15008	0.1512	0.8208	0.7842	0.8866	0.5077	0.001
0.079	29.0	15544	0.1466	0.8222	0.7879	0.8839	0.5109	0.001
0.0803	30.0	16080	0.1479	0.8224	0.7874	0.8962	0.5002	0.001
0.0787	31.0	16616	0.1639	0.8014	0.7579	0.8601	0.4948	0.001
0.0807	32.0	17152	0.1468	0.8230	0.7924	0.8919	0.4966	0.001
0.0776	33.0	17688	0.1480	0.8220	0.7917	0.9005	0.4995	0.001
0.0802	34.0	18224	0.1438	0.8228	0.7907	0.8971	0.5030	0.001
0.0797	35.0	18760	0.1497	0.8206	0.7854	0.8899	0.4909	0.001
0.0781	36.0	19296	0.1407	0.8267	0.7947	0.8933	0.5109	0.001
0.0791	37.0	19832	0.1468	0.8219	0.7714	0.8895	0.5134	0.001
0.082	38.0	20368	0.1538	0.8105	0.7883	0.8863	0.4859	0.001
0.0781	39.0	20904	0.1463	0.8209	0.7858	0.8920	0.5055	0.001
0.0811	40.0	21440	0.1469	0.8151	0.7790	0.8880	0.4977	0.001
0.0786	41.0	21976	0.1518	0.8167	0.7690	0.8872	0.5052	0.001
0.0775	42.0	22512	0.1422	0.8260	0.7913	0.8965	0.5130	0.001
0.0641	43.0	23048	0.1319	0.8340	0.8001	0.8963	0.5248	0.0001
0.0633	44.0	23584	0.1313	0.8326	0.7959	0.8928	0.5298	0.0001
0.0627	45.0	24120	0.1314	0.8324	0.7994	0.8955	0.5241	0.0001
0.0627	46.0	24656	0.1308	0.8324	0.8009	0.8955	0.5234	0.0001
0.0619	47.0	25192	0.1308	0.8333	0.7996	0.8959	0.5252	0.0001
0.0626	48.0	25728	0.1310	0.8333	0.8009	0.8967	0.5198	0.0001
0.063	49.0	26264	0.1311	0.8328	0.7989	0.8957	0.5198	0.0001
0.0623	50.0	26800	0.1308	0.8330	0.7990	0.8962	0.5234	0.0001
0.0627	51.0	27336	0.1309	0.8329	0.8008	0.8972	0.5220	0.0001
0.0624	52.0	27872	0.1305	0.8309	0.7965	0.8909	0.5255	0.0001
0.0626	53.0	28408	0.1307	0.8313	0.7992	0.8947	0.5230	0.0001
0.0621	54.0	28944	0.1304	0.8319	0.7955	0.8964	0.5223	0.0001
0.0631	55.0	29480	0.1299	0.8328	0.8001	0.8949	0.5248	0.0001
0.063	56.0	30016	0.1302	0.8321	0.7989	0.8956	0.5223	0.0001
0.0621	57.0	30552	0.1304	0.8290	0.7970	0.8909	0.5230	0.0001
0.0623	58.0	31088	0.1305	0.8302	0.7978	0.8906	0.5238	0.0001
0.0622	59.0	31624	0.1307	0.8308	0.7965	0.8915	0.5238	0.0001
0.0627	60.0	32160	0.1294	0.8327	0.7998	0.8944	0.5302	0.0001
0.0627	61.0	32696	0.1303	0.8319	0.8000	0.8956	0.5241	0.0001
0.0626	62.0	33232	0.1301	0.8317	0.7982	0.8904	0.5245	0.0001
0.0629	63.0	33768	0.1297	0.8322	0.7989	0.8949	0.5248	0.0001
0.0617	64.0	34304	0.1300	0.8311	0.7982	0.8920	0.5245	0.0001
0.0631	65.0	34840	0.1292	0.8319	0.7986	0.8930	0.5245	0.0001
0.0619	66.0	35376	0.1298	0.8319	0.7982	0.8922	0.5298	0.0001
0.0636	67.0	35912	0.1298	0.8324	0.7999	0.8980	0.5245	0.0001
0.0627	68.0	36448	0.1298	0.8319	0.8006	0.8985	0.5195	0.0001
0.0624	69.0	36984	0.1293	0.8309	0.7980	0.8925	0.5259	0.0001
0.0625	70.0	37520	0.1305	0.8313	0.7967	0.8939	0.5245	0.0001
0.0624	71.0	38056	0.1303	0.8284	0.7942	0.8901	0.5166	0.0001
0.0618	72.0	38592	0.1288	0.8333	0.8010	0.8947	0.5266	1e-05
0.0615	73.0	39128	0.1288	0.8324	0.7990	0.8930	0.5291	1e-05
0.0602	74.0	39664	0.1287	0.8323	0.7989	0.8937	0.5252	1e-05
0.0612	75.0	40200	0.1286	0.8326	0.8004	0.8946	0.5263	1e-05
0.0611	76.0	40736	0.1286	0.8324	0.8001	0.8948	0.5259	1e-05
0.061	77.0	41272	0.1287	0.8320	0.7994	0.8937	0.5280	1e-05
0.0603	78.0	41808	0.1287	0.8323	0.7996	0.8933	0.5277	1e-05
0.0616	79.0	42344	0.1286	0.8322	0.7994	0.8936	0.5270	1e-05
0.061	80.0	42880	0.1286	0.8319	0.7987	0.8934	0.5280	1e-05
0.0607	81.0	43416	0.1287	0.8328	0.8003	0.8938	0.5280	1e-05
0.0609	82.0	43952	0.1288	0.8321	0.7991	0.8935	0.5288	1e-05
0.0611	83.0	44488	0.1287	0.8324	0.7994	0.8937	0.5288	0.0000
0.0611	84.0	45024	0.1286	0.8325	0.7993	0.8936	0.5284	0.0000
0.0608	85.0	45560	0.1286	0.8324	0.7992	0.8935	0.5288	0.0000
0.0607	86.0	46096	0.1286	0.8324	0.7995	0.8936	0.5284	0.0000

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1