vit-base-patch16-224-dmae-va-U5-100-3i

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5087
• Accuracy: 0.8667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	0.5069	0.8333
0.3296	1.94	15	0.5087	0.8667
0.2919	2.97	23	0.5190	0.8667
0.2572	4.0	31	0.6483	0.7667
0.2572	4.9	38	0.5785	0.8167
0.2229	5.94	46	0.5932	0.8333
0.1799	6.97	54	0.5272	0.85
0.1563	8.0	62	0.6124	0.85
0.1563	8.9	69	0.6798	0.8167
0.125	9.94	77	0.7356	0.7833
0.1343	10.97	85	0.5086	0.85
0.0906	12.0	93	0.7601	0.7667
0.103	12.9	100	0.8084	0.8
0.103	13.94	108	0.5612	0.85
0.1002	14.97	116	0.6454	0.8333
0.1107	16.0	124	0.7783	0.8
0.1036	16.9	131	0.7857	0.7833
0.1036	17.94	139	0.6504	0.8167
0.1248	18.97	147	0.6510	0.8167
0.1074	20.0	155	0.7813	0.7833
0.1038	20.9	162	0.6553	0.8
0.1052	21.94	170	0.6449	0.8333
0.1052	22.97	178	0.7444	0.8
0.0782	24.0	186	1.0751	0.6833
0.0952	24.9	193	0.6453	0.8333
0.0803	25.94	201	0.7794	0.8
0.0803	26.97	209	0.6160	0.8333
0.0947	28.0	217	0.6362	0.85
0.0702	28.9	224	0.7610	0.8167
0.0737	29.94	232	0.7924	0.8167
0.0644	30.97	240	0.9755	0.8
0.0644	32.0	248	0.8580	0.8333
0.0695	32.9	255	1.1410	0.7167
0.09	33.94	263	0.8442	0.8
0.0619	34.97	271	1.1689	0.7167
0.0619	36.0	279	0.7599	0.8333
0.0607	36.9	286	0.8498	0.8167
0.0509	37.94	294	0.8331	0.85
0.0666	38.97	302	0.8166	0.8167
0.0615	40.0	310	0.9394	0.7667
0.0615	40.9	317	0.8837	0.8
0.0503	41.94	325	0.8208	0.8333
0.0431	42.97	333	1.1271	0.75
0.0548	44.0	341	0.9044	0.7833
0.0548	44.9	348	0.9017	0.8
0.0414	45.94	356	1.1390	0.75
0.0609	46.97	364	0.8937	0.8
0.0556	48.0	372	0.8459	0.8
0.0556	48.9	379	1.0285	0.7667
0.0417	49.94	387	0.7379	0.85
0.0409	50.97	395	0.7817	0.8333
0.0206	52.0	403	0.7860	0.8167
0.0414	52.9	410	0.8414	0.8167
0.0414	53.94	418	0.8657	0.8
0.0329	54.97	426	0.8824	0.8
0.0394	56.0	434	0.7990	0.8333
0.0373	56.9	441	0.8101	0.8167
0.0373	57.94	449	0.8535	0.8
0.0418	58.97	457	0.9149	0.8167
0.0365	60.0	465	0.9278	0.8
0.0367	60.9	472	0.9064	0.8
0.0355	61.94	480	0.9610	0.7833
0.0355	62.97	488	0.9174	0.8167
0.0492	64.0	496	0.9877	0.7667
0.0326	64.9	503	1.0192	0.7833
0.0233	65.94	511	0.9588	0.8
0.0233	66.97	519	0.9829	0.7833
0.0251	68.0	527	1.0540	0.7667
0.0283	68.9	534	1.0556	0.7667
0.0307	69.94	542	1.0036	0.7833
0.0319	70.97	550	0.9294	0.8
0.0319	72.0	558	1.0077	0.8
0.0246	72.9	565	1.0298	0.7833
0.0205	73.94	573	1.0041	0.7833
0.0345	74.97	581	0.9182	0.7833
0.0345	76.0	589	0.9054	0.8333
0.0181	76.9	596	0.9338	0.8333
0.0287	77.94	604	0.9678	0.7833
0.0268	78.97	612	0.9841	0.7833
0.0293	80.0	620	1.0380	0.7667
0.0293	80.9	627	1.0837	0.7833
0.0222	81.94	635	1.0132	0.7667
0.033	82.97	643	0.9785	0.8
0.0227	84.0	651	0.9848	0.8
0.0227	84.9	658	0.9780	0.8
0.0295	85.94	666	0.9613	0.8167
0.0291	86.97	674	0.9753	0.8167
0.031	88.0	682	0.9831	0.8
0.031	88.9	689	0.9820	0.8
0.0233	89.94	697	0.9793	0.8
0.0195	90.32	700	0.9788	0.8

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0