swiftformer-xs-ve-U13-b-80c

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.7032
• Accuracy: 0.8043

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.15
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.3860	0.2391
1.3859	2.0	13	1.3844	0.3043
1.3859	2.92	19	1.3820	0.1957
1.381	4.0	26	1.3746	0.1739
1.3573	4.92	32	1.3643	0.1957
1.3573	6.0	39	1.3561	0.1522
1.2692	6.92	45	1.3583	0.1522
1.1682	8.0	52	1.3623	0.1739
1.1682	8.92	58	1.3296	0.2609
1.1005	10.0	65	1.2663	0.3913
0.9884	10.92	71	1.3160	0.3696
0.9884	12.0	78	1.1806	0.4783
0.9111	12.92	84	1.1560	0.6087
0.8464	14.0	91	1.1350	0.5870
0.8464	14.92	97	1.0768	0.6304
0.7768	16.0	104	0.9707	0.6087
0.6754	16.92	110	0.9544	0.6522
0.6754	18.0	117	0.9885	0.6739
0.657	18.92	123	0.8578	0.6957
0.5408	20.0	130	0.7794	0.7391
0.5408	20.92	136	0.8072	0.7391
0.5094	22.0	143	0.7917	0.6739
0.5094	22.92	149	0.7975	0.6739
0.4546	24.0	156	0.7583	0.7609
0.3722	24.92	162	0.7074	0.7826
0.3722	26.0	169	0.6909	0.7391
0.3494	26.92	175	0.7032	0.8043
0.3092	28.0	182	0.8149	0.7826
0.3092	28.92	188	0.7898	0.7826
0.2643	30.0	195	0.7312	0.8043
0.2659	30.92	201	0.7598	0.7174
0.2659	32.0	208	0.7531	0.7609
0.2298	32.92	214	0.6877	0.8043
0.2147	34.0	221	0.6864	0.8043
0.2147	34.92	227	0.7656	0.7391
0.2457	36.0	234	0.8494	0.7391
0.1905	36.92	240	0.7319	0.7609
0.1905	38.0	247	0.8290	0.6957
0.2073	38.92	253	0.7963	0.7609
0.1603	40.0	260	0.8693	0.6957
0.1603	40.92	266	0.7138	0.8043
0.1852	42.0	273	0.7274	0.7609
0.1852	42.92	279	0.8353	0.6739
0.1641	44.0	286	0.9382	0.6957
0.1568	44.92	292	0.8655	0.7174
0.1568	46.0	299	0.7621	0.7391
0.1498	46.92	305	0.7944	0.7174
0.1563	48.0	312	0.8433	0.6957
0.1563	48.92	318	0.8633	0.7609
0.1554	50.0	325	0.8543	0.7391
0.1316	50.92	331	0.9127	0.7174
0.1316	52.0	338	0.9248	0.6957
0.1264	52.92	344	0.9349	0.6957
0.1082	54.0	351	0.9785	0.6739
0.1082	54.92	357	1.0165	0.6739
0.1366	56.0	364	0.8369	0.6957
0.1546	56.92	370	0.8372	0.7174
0.1546	58.0	377	0.8596	0.6957
0.1218	58.92	383	0.8054	0.7174
0.1162	60.0	390	0.7963	0.7391
0.1162	60.92	396	0.7953	0.7391
0.0876	62.0	403	0.8229	0.7391
0.0876	62.92	409	0.8365	0.7391
0.1032	64.0	416	0.8162	0.7609
0.0825	64.92	422	0.8646	0.7391
0.0825	66.0	429	0.9135	0.7391
0.1119	66.92	435	0.9164	0.7391
0.0949	68.0	442	0.9232	0.7391
0.0949	68.92	448	0.9381	0.7391
0.1227	70.0	455	0.8998	0.7391
0.0872	70.92	461	0.9632	0.7174
0.0872	72.0	468	0.8566	0.7174
0.1033	72.92	474	0.8909	0.7174
0.0876	73.85	480	0.8869	0.7609

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0