beit-base-patch16-224-dmae-va-U5-42

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0039
• Accuracy: 0.8167

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.3471	0.4667
1.6023	1.94	15	1.0873	0.5833
1.1509	2.97	23	0.9948	0.5833
0.826	4.0	31	0.7244	0.7167
0.826	4.9	38	0.5741	0.7333
0.5551	5.94	46	0.6569	0.75
0.3649	6.97	54	0.6322	0.7167
0.2592	8.0	62	0.6994	0.7333
0.2592	8.9	69	0.6590	0.7333
0.1958	9.94	77	0.6846	0.7667
0.1664	10.97	85	0.7166	0.7667
0.1571	12.0	93	0.7842	0.7833
0.1174	12.9	100	0.8465	0.8
0.1174	13.94	108	0.9116	0.7667
0.0956	14.97	116	0.9741	0.75
0.1252	16.0	124	0.7760	0.8
0.0933	16.9	131	0.9424	0.7833
0.0933	17.94	139	1.0445	0.7333
0.1455	18.97	147	0.8525	0.7333
0.1034	20.0	155	0.8222	0.7667
0.0855	20.9	162	0.8991	0.7833
0.0985	21.94	170	0.8955	0.8
0.0985	22.97	178	0.9603	0.7667
0.087	24.0	186	0.9932	0.7833
0.0832	24.9	193	1.0100	0.7833
0.0632	25.94	201	0.9393	0.7667
0.0632	26.97	209	0.9062	0.7833
0.0778	28.0	217	0.9339	0.8
0.0627	28.9	224	1.0039	0.8167
0.0837	29.94	232	1.0636	0.7333
0.0595	30.97	240	1.0424	0.75
0.0595	32.0	248	1.0514	0.8
0.0706	32.9	255	1.0639	0.7833
0.0565	33.94	263	1.0494	0.7667
0.0515	34.97	271	1.0628	0.7667
0.0515	36.0	279	1.1089	0.7667
0.0614	36.9	286	1.0861	0.8
0.0496	37.94	294	1.0713	0.8

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2