vit-base_rvl-cdip-tiny_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2802
• Accuracy: 0.5747
• Brier Loss: 0.6822
• Nll: 3.2886
• F1 Micro: 0.5747
• F1 Macro: 0.5757
• Ece: 0.2786
• Aurc: 0.2132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
No log	1.0	250	4.1512	0.1727	0.9045	5.5051	0.1727	0.0947	0.0704	0.7164
4.2402	2.0	500	3.8933	0.216	0.8775	4.1816	0.216	0.1697	0.0699	0.6624
4.2402	3.0	750	3.4256	0.3207	0.8113	3.6783	0.3207	0.2567	0.0645	0.5125
3.5189	4.0	1000	3.1611	0.3673	0.7763	3.6447	0.3673	0.3039	0.0797	0.4450
3.5189	5.0	1250	2.7791	0.4253	0.7216	3.1536	0.4253	0.3860	0.0982	0.3729
2.7963	6.0	1500	2.6525	0.4323	0.7004	3.0187	0.4323	0.4117	0.0992	0.3440
2.7963	7.0	1750	2.3623	0.5005	0.6489	2.8371	0.5005	0.4747	0.1076	0.2843
2.3741	8.0	2000	2.4259	0.4798	0.6704	2.9344	0.4798	0.4680	0.1164	0.3045
2.3741	9.0	2250	2.3034	0.5005	0.6431	2.8598	0.5005	0.4892	0.1306	0.2683
2.0855	10.0	2500	2.1550	0.5298	0.6264	2.6847	0.5298	0.5164	0.1413	0.2480
2.0855	11.0	2750	2.0891	0.5455	0.6162	2.6978	0.5455	0.5330	0.1428	0.2343
1.8265	12.0	3000	2.2045	0.5252	0.6627	2.7900	0.5252	0.5045	0.1997	0.2507
1.8265	13.0	3250	2.0080	0.5597	0.5948	2.7128	0.5597	0.5564	0.1389	0.2145
1.6099	14.0	3500	2.1966	0.5353	0.6594	2.8505	0.5353	0.5198	0.1984	0.2581
1.6099	15.0	3750	2.0788	0.547	0.6191	2.7214	0.547	0.5419	0.1729	0.2294
1.4149	16.0	4000	2.0634	0.5485	0.6235	2.7486	0.5485	0.5491	0.1872	0.2225
1.4149	17.0	4250	2.0722	0.5597	0.6241	2.7989	0.5597	0.5574	0.1912	0.2189
1.2282	18.0	4500	2.1226	0.557	0.6327	2.9138	0.557	0.5584	0.2016	0.2205
1.2282	19.0	4750	2.1013	0.5577	0.6326	2.8846	0.5577	0.5574	0.2051	0.2200
1.0543	20.0	5000	2.1902	0.5637	0.6519	2.9362	0.5637	0.5556	0.2261	0.2273
1.0543	21.0	5250	2.2291	0.5603	0.6620	2.9256	0.5603	0.5532	0.2469	0.2350
0.8882	22.0	5500	2.2152	0.5605	0.6613	3.0823	0.5605	0.5563	0.2397	0.2234
0.8882	23.0	5750	2.2309	0.5617	0.6600	3.1164	0.5617	0.5571	0.2520	0.2252
0.7308	24.0	6000	2.2332	0.5655	0.6631	3.1202	0.5655	0.5661	0.2502	0.2241
0.7308	25.0	6250	2.3018	0.5663	0.6762	3.2623	0.5663	0.5652	0.2640	0.2265
0.6001	26.0	6500	2.3505	0.5547	0.6923	3.3289	0.5547	0.5592	0.2790	0.2279
0.6001	27.0	6750	2.3821	0.5555	0.6932	3.4374	0.5555	0.5538	0.2827	0.2275
0.4912	28.0	7000	2.3788	0.5675	0.6915	3.3014	0.5675	0.5637	0.2865	0.2324
0.4912	29.0	7250	2.4068	0.556	0.7028	3.4904	0.556	0.5559	0.2906	0.2365
0.4068	30.0	7500	2.4476	0.5557	0.7044	3.4350	0.5557	0.5572	0.2846	0.2387
0.4068	31.0	7750	2.4179	0.562	0.7021	3.4782	0.562	0.5619	0.2911	0.2305
0.3364	32.0	8000	2.3915	0.5615	0.6961	3.4704	0.5615	0.5623	0.2889	0.2294
0.3364	33.0	8250	2.3860	0.568	0.6957	3.4578	0.568	0.5703	0.2869	0.2263
0.2862	34.0	8500	2.4250	0.5647	0.7022	3.4923	0.5647	0.5638	0.2928	0.2282
0.2862	35.0	8750	2.4453	0.5587	0.7106	3.6175	0.5587	0.5594	0.2970	0.2306
0.2397	36.0	9000	2.3919	0.5653	0.6964	3.4399	0.5653	0.5675	0.2881	0.2197
0.2397	37.0	9250	2.3870	0.5647	0.6995	3.4910	0.5647	0.5657	0.2941	0.2237
0.2058	38.0	9500	2.4080	0.5663	0.7033	3.5314	0.5663	0.5673	0.2979	0.2271
0.2058	39.0	9750	2.3727	0.5675	0.6975	3.3806	0.5675	0.5708	0.2930	0.2240
0.1819	40.0	10000	2.3627	0.5745	0.6913	3.4237	0.5745	0.5751	0.2847	0.2217
0.1819	41.0	10250	2.3497	0.564	0.6952	3.3908	0.564	0.5626	0.2931	0.2208
0.1587	42.0	10500	2.3168	0.5705	0.6842	3.3858	0.5705	0.5725	0.2808	0.2181
0.1587	43.0	10750	2.2910	0.5715	0.6768	3.3739	0.5715	0.5727	0.2777	0.2127
0.1402	44.0	11000	2.3053	0.5713	0.6808	3.4128	0.5713	0.5724	0.2793	0.2133
0.1402	45.0	11250	2.3029	0.5743	0.6848	3.3133	0.5743	0.5750	0.2771	0.2192
0.1257	46.0	11500	2.2965	0.5695	0.6856	3.2338	0.5695	0.5697	0.2858	0.2158
0.1257	47.0	11750	2.2823	0.5685	0.6847	3.2705	0.5685	0.5693	0.2828	0.2153
0.1134	48.0	12000	2.2800	0.5753	0.6803	3.2797	0.5753	0.5759	0.2795	0.2139
0.1134	49.0	12250	2.2766	0.5733	0.6823	3.2828	0.5733	0.5751	0.2777	0.2135
0.1039	50.0	12500	2.2802	0.5747	0.6822	3.2886	0.5747	0.5757	0.2786	0.2132

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1.post200
• Datasets 2.9.0
• Tokenizers 0.13.2