swin-tiny-patch4-window7-224-dmae-va-U5-42

This model is a fine-tuned version of microsoft/swin-tiny-patch4-window7-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6653
• Accuracy: 0.75

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.3831	0.45
1.3852	1.94	15	1.3624	0.45
1.3728	2.97	23	1.2927	0.4667
1.2791	4.0	31	1.1182	0.4833
1.2791	4.9	38	1.0065	0.4667
1.094	5.94	46	0.8931	0.6
0.9601	6.97	54	0.9210	0.4833
0.8598	8.0	62	0.9478	0.5167
0.8598	8.9	69	0.8558	0.5833
0.7558	9.94	77	0.9259	0.65
0.6696	10.97	85	0.7953	0.6167
0.6079	12.0	93	0.7281	0.7
0.516	12.9	100	0.8552	0.6667
0.516	13.94	108	0.6653	0.75
0.4475	14.97	116	0.7548	0.6833
0.4152	16.0	124	0.7556	0.7167
0.3759	16.9	131	0.7038	0.7333
0.3759	17.94	139	0.7356	0.7167
0.3366	18.97	147	0.6650	0.75
0.3212	20.0	155	0.7668	0.7167
0.2903	20.9	162	0.7997	0.6833
0.312	21.94	170	0.7473	0.7167
0.312	22.97	178	0.8479	0.7
0.2488	24.0	186	0.8070	0.7
0.283	24.9	193	0.8080	0.6833
0.2109	25.94	201	0.8220	0.7
0.2109	26.97	209	0.8354	0.7167
0.2215	28.0	217	0.8668	0.7
0.2067	28.9	224	0.8479	0.75
0.1967	29.94	232	0.8868	0.7167
0.1948	30.97	240	0.8883	0.7
0.1948	32.0	248	0.8612	0.7333
0.186	32.9	255	0.8860	0.7333
0.1662	33.94	263	0.9057	0.75
0.1773	34.97	271	0.9140	0.7167
0.1773	36.0	279	0.9013	0.7333
0.1519	36.9	286	0.8869	0.75
0.1775	37.94	294	0.8840	0.7333

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0