beit-base-patch16-224-OT-3

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3932
• Accuracy: 0.8710

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.91	5	1.3788	0.5
1.3775	2.0	11	1.3397	0.5161
1.3775	2.91	16	1.2780	0.5161
1.2993	4.0	22	1.1642	0.6774
1.2993	4.91	27	1.0752	0.7097
1.1713	6.0	33	0.9749	0.7258
1.1713	6.91	38	0.8666	0.7581
0.9956	8.0	44	0.7634	0.8065
0.9956	8.91	49	0.6863	0.8226
0.845	10.0	55	0.6005	0.8226
0.7145	10.91	60	0.5364	0.8387
0.7145	12.0	66	0.5585	0.8065
0.5907	12.91	71	0.4962	0.7742
0.5907	14.0	77	0.5070	0.7581
0.5531	14.91	82	0.4648	0.8226
0.5531	16.0	88	0.4812	0.7581
0.4878	16.91	93	0.4281	0.8226
0.4878	18.0	99	0.4796	0.7419
0.4583	18.91	104	0.3913	0.8226
0.4546	20.0	110	0.4085	0.7742
0.4546	20.91	115	0.4016	0.8387
0.4118	22.0	121	0.4125	0.8226
0.4118	22.91	126	0.4282	0.8226
0.3939	24.0	132	0.4869	0.7742
0.3939	24.91	137	0.3723	0.8548
0.4138	26.0	143	0.4032	0.8065
0.4138	26.91	148	0.4397	0.8065
0.3599	28.0	154	0.3714	0.8548
0.3599	28.91	159	0.3800	0.8548
0.3629	30.0	165	0.4158	0.8065
0.336	30.91	170	0.4100	0.8226
0.336	32.0	176	0.4001	0.8387
0.3306	32.91	181	0.3925	0.8548
0.3306	34.0	187	0.3932	0.8710
0.3319	34.91	192	0.3942	0.8710
0.3319	36.0	198	0.3883	0.8710
0.3324	36.36	200	0.3886	0.8710

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0