vit-small_tobacco3482_kd_CEKD_t1.5_a0.9

This model is a fine-tuned version of WinKawaks/vit-small-patch16-224 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5492
• Accuracy: 0.84
• Brier Loss: 0.2438
• Nll: 1.0175
• F1 Micro: 0.8400
• F1 Macro: 0.8329
• Ece: 0.1581
• Aurc: 0.0460

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
No log	1.0	7	2.1371	0.215	0.8750	5.2661	0.2150	0.1261	0.2616	0.6901
No log	2.0	14	1.7146	0.4	0.7405	3.6392	0.4000	0.2249	0.2801	0.4047
No log	3.0	21	1.1877	0.625	0.5608	2.0254	0.625	0.5681	0.3161	0.2040
No log	4.0	28	0.8633	0.715	0.4058	1.6421	0.715	0.6656	0.2020	0.1142
No log	5.0	35	0.8597	0.72	0.3947	1.6962	0.72	0.7299	0.2181	0.1133
No log	6.0	42	0.7266	0.785	0.3157	1.6428	0.785	0.7648	0.2063	0.0758
No log	7.0	49	0.7662	0.77	0.3428	1.4695	0.7700	0.7666	0.1871	0.0998
No log	8.0	56	0.7824	0.77	0.3365	1.5995	0.7700	0.7346	0.1840	0.0980
No log	9.0	63	0.7245	0.805	0.3102	1.2669	0.805	0.8012	0.1789	0.0855
No log	10.0	70	0.6787	0.8	0.2944	1.3351	0.8000	0.7754	0.1578	0.0665
No log	11.0	77	0.6497	0.805	0.2870	1.3980	0.805	0.8029	0.1743	0.0709
No log	12.0	84	0.6353	0.82	0.2747	1.3397	0.82	0.8085	0.1670	0.0687
No log	13.0	91	0.7204	0.79	0.3163	1.4500	0.79	0.7945	0.1660	0.0782
No log	14.0	98	0.6632	0.825	0.2714	1.5658	0.825	0.8110	0.1827	0.0726
No log	15.0	105	0.6417	0.8	0.2840	1.3774	0.8000	0.7984	0.1618	0.0703
No log	16.0	112	0.5899	0.825	0.2687	1.0331	0.825	0.8220	0.1569	0.0603
No log	17.0	119	0.5924	0.83	0.2508	1.4167	0.83	0.8182	0.1414	0.0549
No log	18.0	126	0.5885	0.825	0.2608	1.1991	0.825	0.8174	0.1677	0.0607
No log	19.0	133	0.5898	0.82	0.2634	1.2879	0.82	0.8145	0.1563	0.0610
No log	20.0	140	0.5509	0.825	0.2439	1.1130	0.825	0.8127	0.1532	0.0475
No log	21.0	147	0.5719	0.82	0.2585	1.1331	0.82	0.8101	0.1640	0.0490
No log	22.0	154	0.5650	0.85	0.2449	1.2095	0.85	0.8429	0.1622	0.0595
No log	23.0	161	0.5538	0.83	0.2492	1.0979	0.83	0.8227	0.1759	0.0515
No log	24.0	168	0.5514	0.84	0.2396	1.1748	0.8400	0.8360	0.1449	0.0479
No log	25.0	175	0.5549	0.815	0.2497	1.0876	0.815	0.8080	0.1668	0.0502
No log	26.0	182	0.5469	0.84	0.2397	1.1651	0.8400	0.8317	0.1560	0.0471
No log	27.0	189	0.5584	0.84	0.2508	1.0605	0.8400	0.8253	0.1801	0.0486
No log	28.0	196	0.5395	0.845	0.2371	1.0749	0.845	0.8302	0.1448	0.0438
No log	29.0	203	0.5478	0.84	0.2436	1.0599	0.8400	0.8271	0.1556	0.0470
No log	30.0	210	0.5432	0.835	0.2402	1.0595	0.835	0.8206	0.1613	0.0457
No log	31.0	217	0.5454	0.83	0.2422	1.0518	0.83	0.8176	0.1556	0.0462
No log	32.0	224	0.5456	0.83	0.2415	1.0500	0.83	0.8176	0.1555	0.0461
No log	33.0	231	0.5471	0.835	0.2430	1.0492	0.835	0.8233	0.1616	0.0466
No log	34.0	238	0.5456	0.83	0.2424	1.0495	0.83	0.8176	0.1636	0.0467
No log	35.0	245	0.5482	0.835	0.2434	1.0438	0.835	0.8239	0.1717	0.0474
No log	36.0	252	0.5462	0.835	0.2425	1.0461	0.835	0.8239	0.1507	0.0462
No log	37.0	259	0.5488	0.83	0.2435	1.0468	0.83	0.8176	0.1377	0.0471
No log	38.0	266	0.5461	0.84	0.2420	1.0389	0.8400	0.8296	0.1379	0.0458
No log	39.0	273	0.5458	0.84	0.2423	1.0387	0.8400	0.8296	0.1545	0.0457
No log	40.0	280	0.5483	0.835	0.2435	1.0382	0.835	0.8233	0.1343	0.0466
No log	41.0	287	0.5475	0.835	0.2430	1.0378	0.835	0.8233	0.1408	0.0454
No log	42.0	294	0.5463	0.835	0.2424	1.0368	0.835	0.8233	0.1463	0.0454
No log	43.0	301	0.5467	0.835	0.2428	1.0335	0.835	0.8233	0.1453	0.0458
No log	44.0	308	0.5470	0.835	0.2429	1.0331	0.835	0.8233	0.1597	0.0459
No log	45.0	315	0.5469	0.835	0.2426	1.0336	0.835	0.8233	0.1487	0.0459
No log	46.0	322	0.5473	0.835	0.2431	1.0322	0.835	0.8233	0.1486	0.0465
No log	47.0	329	0.5464	0.84	0.2425	1.0324	0.8400	0.8329	0.1443	0.0454
No log	48.0	336	0.5462	0.835	0.2426	1.0298	0.835	0.8233	0.1527	0.0454
No log	49.0	343	0.5471	0.835	0.2427	1.0305	0.835	0.8233	0.1619	0.0456
No log	50.0	350	0.5479	0.84	0.2433	1.0304	0.8400	0.8329	0.1549	0.0457
No log	51.0	357	0.5471	0.835	0.2427	1.0296	0.835	0.8233	0.1607	0.0458
No log	52.0	364	0.5475	0.835	0.2431	1.0282	0.835	0.8233	0.1596	0.0458
No log	53.0	371	0.5474	0.84	0.2428	1.0294	0.8400	0.8329	0.1603	0.0457
No log	54.0	378	0.5482	0.835	0.2436	1.0263	0.835	0.8233	0.1460	0.0461
No log	55.0	385	0.5468	0.84	0.2424	1.0264	0.8400	0.8329	0.1491	0.0454
No log	56.0	392	0.5479	0.84	0.2432	1.0263	0.8400	0.8329	0.1594	0.0452
No log	57.0	399	0.5467	0.84	0.2426	1.0259	0.8400	0.8329	0.1476	0.0454
No log	58.0	406	0.5484	0.835	0.2434	1.0237	0.835	0.8233	0.1379	0.0463
No log	59.0	413	0.5473	0.835	0.2429	1.0245	0.835	0.8233	0.1521	0.0458
No log	60.0	420	0.5475	0.835	0.2430	1.0240	0.835	0.8233	0.1523	0.0458
No log	61.0	427	0.5475	0.835	0.2430	1.0239	0.835	0.8233	0.1438	0.0461
No log	62.0	434	0.5476	0.835	0.2430	1.0227	0.835	0.8233	0.1522	0.0461
No log	63.0	441	0.5478	0.835	0.2430	1.0235	0.835	0.8233	0.1520	0.0460
No log	64.0	448	0.5478	0.84	0.2432	1.0215	0.8400	0.8329	0.1576	0.0458
No log	65.0	455	0.5478	0.835	0.2430	1.0229	0.835	0.8233	0.1592	0.0461
No log	66.0	462	0.5481	0.84	0.2433	1.0219	0.8400	0.8329	0.1582	0.0459
No log	67.0	469	0.5482	0.84	0.2434	1.0214	0.8400	0.8329	0.1665	0.0456
No log	68.0	476	0.5482	0.835	0.2433	1.0209	0.835	0.8233	0.1445	0.0463
No log	69.0	483	0.5484	0.84	0.2435	1.0210	0.8400	0.8329	0.1578	0.0458
No log	70.0	490	0.5479	0.84	0.2433	1.0206	0.8400	0.8329	0.1662	0.0457
No log	71.0	497	0.5486	0.84	0.2435	1.0210	0.8400	0.8329	0.1401	0.0460
0.1783	72.0	504	0.5489	0.84	0.2437	1.0204	0.8400	0.8329	0.1581	0.0460
0.1783	73.0	511	0.5483	0.835	0.2435	1.0194	0.835	0.8233	0.1712	0.0460
0.1783	74.0	518	0.5489	0.84	0.2437	1.0198	0.8400	0.8329	0.1668	0.0461
0.1783	75.0	525	0.5486	0.84	0.2435	1.0194	0.8400	0.8329	0.1666	0.0458
0.1783	76.0	532	0.5487	0.84	0.2436	1.0194	0.8400	0.8329	0.1710	0.0458
0.1783	77.0	539	0.5485	0.84	0.2434	1.0191	0.8400	0.8329	0.1392	0.0459
0.1783	78.0	546	0.5486	0.84	0.2435	1.0191	0.8400	0.8329	0.1579	0.0458
0.1783	79.0	553	0.5486	0.84	0.2436	1.0190	0.8400	0.8329	0.1582	0.0459
0.1783	80.0	560	0.5492	0.84	0.2438	1.0194	0.8400	0.8329	0.1581	0.0461
0.1783	81.0	567	0.5486	0.84	0.2435	1.0189	0.8400	0.8329	0.1581	0.0460
0.1783	82.0	574	0.5489	0.84	0.2437	1.0185	0.8400	0.8329	0.1581	0.0460
0.1783	83.0	581	0.5491	0.84	0.2438	1.0188	0.8400	0.8329	0.1574	0.0460
0.1783	84.0	588	0.5490	0.84	0.2438	1.0183	0.8400	0.8329	0.1581	0.0461
0.1783	85.0	595	0.5491	0.84	0.2438	1.0184	0.8400	0.8329	0.1485	0.0461
0.1783	86.0	602	0.5492	0.84	0.2439	1.0177	0.8400	0.8329	0.1584	0.0461
0.1783	87.0	609	0.5491	0.84	0.2438	1.0180	0.8400	0.8329	0.1582	0.0461
0.1783	88.0	616	0.5493	0.84	0.2438	1.0180	0.8400	0.8329	0.1584	0.0462
0.1783	89.0	623	0.5493	0.84	0.2438	1.0178	0.8400	0.8329	0.1584	0.0462
0.1783	90.0	630	0.5490	0.84	0.2437	1.0180	0.8400	0.8329	0.1584	0.0461
0.1783	91.0	637	0.5491	0.84	0.2438	1.0177	0.8400	0.8329	0.1581	0.0459
0.1783	92.0	644	0.5492	0.84	0.2438	1.0177	0.8400	0.8329	0.1582	0.0461
0.1783	93.0	651	0.5491	0.84	0.2437	1.0180	0.8400	0.8329	0.1581	0.0460
0.1783	94.0	658	0.5491	0.84	0.2438	1.0180	0.8400	0.8329	0.1584	0.0461
0.1783	95.0	665	0.5492	0.84	0.2438	1.0177	0.8400	0.8329	0.1582	0.0461
0.1783	96.0	672	0.5492	0.84	0.2438	1.0176	0.8400	0.8329	0.1582	0.0461
0.1783	97.0	679	0.5492	0.84	0.2438	1.0175	0.8400	0.8329	0.1581	0.0460
0.1783	98.0	686	0.5491	0.84	0.2438	1.0175	0.8400	0.8329	0.1582	0.0461
0.1783	99.0	693	0.5491	0.84	0.2438	1.0175	0.8400	0.8329	0.1580	0.0460
0.1783	100.0	700	0.5492	0.84	0.2438	1.0175	0.8400	0.8329	0.1581	0.0460

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1.post200
• Datasets 2.9.0
• Tokenizers 0.13.2