vit-base-patch16-224-in21k-finetuned-lora-ISIC-2019

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5908
• Accuracy: 0.8698

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.0273	0.99	62	0.9625	0.6629
0.8456	2.0	125	0.8068	0.6990
0.771	2.99	187	0.7126	0.7362
0.682	4.0	250	0.6901	0.7497
0.641	4.99	312	0.6500	0.7570
0.569	6.0	375	0.6460	0.7638
0.5696	6.99	437	0.5974	0.7796
0.5411	8.0	500	0.6076	0.7796
0.5015	8.99	562	0.5633	0.7880
0.4999	10.0	625	0.5726	0.7892
0.4569	10.99	687	0.5587	0.7993
0.4348	12.0	750	0.5712	0.7999
0.4321	12.99	812	0.5455	0.7971
0.4072	14.0	875	0.5409	0.8083
0.3821	14.99	937	0.5464	0.8106
0.376	16.0	1000	0.5402	0.8151
0.3427	16.99	1062	0.5327	0.8168
0.2938	18.0	1125	0.5301	0.8100
0.3116	18.99	1187	0.5457	0.8134
0.3231	20.0	1250	0.5507	0.8157
0.2942	20.99	1312	0.5307	0.8157
0.299	22.0	1375	0.5178	0.8320
0.2821	22.99	1437	0.5436	0.8241
0.2576	24.0	1500	0.5332	0.8224
0.2728	24.99	1562	0.5401	0.8315
0.2383	26.0	1625	0.5710	0.8343
0.2504	26.99	1687	0.5498	0.8326
0.2474	28.0	1750	0.5372	0.8348
0.2156	28.99	1812	0.5628	0.8309
0.2035	30.0	1875	0.5538	0.8377
0.2043	30.99	1937	0.5485	0.8416
0.1964	32.0	2000	0.5695	0.8360
0.2086	32.99	2062	0.5628	0.8439
0.1893	34.0	2125	0.5583	0.8399
0.1857	34.99	2187	0.5525	0.8388
0.1811	36.0	2250	0.5287	0.8444
0.196	36.99	2312	0.5324	0.8416
0.1644	38.0	2375	0.5433	0.8472
0.1754	38.99	2437	0.5511	0.8478
0.1521	40.0	2500	0.5626	0.8467
0.1536	40.99	2562	0.5634	0.8501
0.1399	42.0	2625	0.5802	0.8596
0.1589	42.99	2687	0.6154	0.8298
0.1575	44.0	2750	0.5630	0.8523
0.1523	44.99	2812	0.5822	0.8489
0.1457	46.0	2875	0.5842	0.8529
0.1326	46.99	2937	0.5729	0.8551
0.1319	48.0	3000	0.5706	0.8546
0.131	48.99	3062	0.5893	0.8551
0.1588	50.0	3125	0.5695	0.8461
0.1297	50.99	3187	0.5902	0.8455
0.1603	52.0	3250	0.5921	0.8450
0.108	52.99	3312	0.6141	0.8478
0.1483	54.0	3375	0.5862	0.8506
0.1191	54.99	3437	0.5707	0.8455
0.1148	56.0	3500	0.5644	0.8636
0.1052	56.99	3562	0.5904	0.8602
0.1307	58.0	3625	0.5818	0.8489
0.1188	58.99	3687	0.5898	0.8489
0.1114	60.0	3750	0.6035	0.8517
0.1055	60.99	3812	0.6122	0.8534
0.1326	62.0	3875	0.6129	0.8540
0.118	62.99	3937	0.5966	0.8529
0.0982	64.0	4000	0.6206	0.8546
0.1021	64.99	4062	0.6053	0.8551
0.0988	66.0	4125	0.6225	0.8495
0.102	66.99	4187	0.6114	0.8579
0.108	68.0	4250	0.6544	0.8461
0.0959	68.99	4312	0.6473	0.8467
0.0988	70.0	4375	0.6325	0.8484
0.0949	70.99	4437	0.6549	0.8472
0.0998	72.0	4500	0.6151	0.8478
0.0861	72.99	4562	0.6141	0.8489
0.099	74.0	4625	0.6109	0.8517
0.0848	74.99	4687	0.6202	0.8478
0.0881	76.0	4750	0.6249	0.8546
0.1046	76.99	4812	0.6102	0.8568
0.0859	78.0	4875	0.6112	0.8625
0.0946	78.99	4937	0.6136	0.8630
0.0902	80.0	5000	0.6027	0.8630
0.093	80.99	5062	0.6099	0.8641
0.0857	82.0	5125	0.5908	0.8698
0.0983	82.99	5187	0.5939	0.8625
0.0819	84.0	5250	0.6139	0.8602
0.0815	84.99	5312	0.6171	0.8636
0.0758	86.0	5375	0.6263	0.8636
0.0856	86.99	5437	0.6137	0.8619
0.0922	88.0	5500	0.6294	0.8647
0.0728	88.99	5562	0.6257	0.8619
0.0791	90.0	5625	0.6168	0.8658
0.0761	90.99	5687	0.6233	0.8675
0.0734	92.0	5750	0.6210	0.8653
0.085	92.99	5812	0.6187	0.8630
0.0816	94.0	5875	0.6183	0.8625
0.0763	94.99	5937	0.6207	0.8687
0.077	96.0	6000	0.6161	0.8664
0.0872	96.99	6062	0.6127	0.8664
0.0741	98.0	6125	0.6152	0.8687
0.0746	98.99	6187	0.6147	0.8670
0.0804	99.2	6200	0.6147	0.8670

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1
• Datasets 2.12.0
• Tokenizers 0.13.2