beit-base-patch16-224-dmae-va-U5-42E

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5797
• Accuracy: 0.8333

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9032	7	1.2066	0.55
1.5038	1.9355	15	1.2167	0.4167
1.1504	2.9677	23	0.8821	0.65
0.8299	4.0	31	0.6489	0.7667
0.8299	4.9032	38	0.5797	0.8333
0.5887	5.9355	46	0.6618	0.75
0.3965	6.9677	54	0.6531	0.7833
0.3089	8.0	62	0.7609	0.7
0.3089	8.9032	69	0.8609	0.6833
0.2393	9.9355	77	0.6910	0.7833
0.1928	10.9677	85	0.7774	0.8
0.1993	12.0	93	0.8424	0.7833
0.165	12.9032	100	0.7478	0.7667
0.165	13.9355	108	0.7573	0.75
0.1117	14.9677	116	0.8059	0.8167
0.1171	16.0	124	0.8982	0.7667
0.0961	16.9032	131	0.9133	0.8
0.0961	17.9355	139	0.9121	0.7667
0.1359	18.9677	147	0.9297	0.8
0.0981	20.0	155	1.0124	0.7333
0.0817	20.9032	162	0.9628	0.75
0.0976	21.9355	170	0.9664	0.7667
0.0976	22.9677	178	0.7980	0.8167
0.0899	24.0	186	0.8366	0.7667
0.1052	24.9032	193	0.9160	0.7667
0.0817	25.9355	201	0.9701	0.7667
0.0817	26.9677	209	0.9995	0.75
0.0886	28.0	217	0.8483	0.8
0.0766	28.9032	224	0.8954	0.7833
0.0923	29.9355	232	0.9606	0.7833
0.0579	30.9677	240	0.9958	0.75
0.0579	32.0	248	0.9665	0.7833
0.0707	32.9032	255	1.0259	0.7667
0.0756	33.9355	263	1.0627	0.75
0.0528	34.9677	271	1.0508	0.7667
0.0528	36.0	279	1.0998	0.7667
0.0706	36.9032	286	1.0694	0.75
0.0658	37.9355	294	1.0561	0.7667

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1