vit-pretraining-2024_03_10

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4444

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.6875e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 200.0

Training results

Training Loss	Epoch	Step	Validation Loss
1.0002	1.0	2443	1.0000
0.9832	2.0	4886	0.9753
0.9246	3.0	7329	0.9304
0.8979	4.0	9772	0.8855
0.8307	5.0	12215	0.8077
0.7861	6.0	14658	0.7776
0.7665	7.0	17101	0.7557
0.7421	8.0	19544	0.7337
0.6841	9.0	21987	0.7133
0.6875	10.0	24430	0.7001
0.6991	11.0	26873	0.6887
0.6991	12.0	29316	0.6711
0.6584	13.0	31759	0.6674
0.6619	14.0	34202	0.6507
0.6389	15.0	36645	0.6462
0.6381	16.0	39088	0.6370
0.616	17.0	41531	0.6248
0.627	18.0	43974	0.6213
0.6179	19.0	46417	0.6150
0.6226	20.0	48860	0.6112
0.5876	21.0	51303	0.6062
0.613	22.0	53746	0.5990
0.5864	23.0	56189	0.5948
0.5741	24.0	58632	0.5940
0.5886	25.0	61075	0.5883
0.6028	26.0	63518	0.5890
0.578	27.0	65961	0.5841
0.5846	28.0	68404	0.5779
0.5725	29.0	70847	0.5766
0.5684	30.0	73290	0.5791
0.5689	31.0	75733	0.5726
0.5478	32.0	78176	0.5708
0.5739	33.0	80619	0.5697
0.5578	34.0	83062	0.5629
0.568	35.0	85505	0.5696
0.5819	36.0	87948	0.5649
0.5442	37.0	90391	0.5649
0.5616	38.0	92834	0.5626
0.5386	39.0	95277	0.5617
0.5725	40.0	97720	0.5552
0.549	41.0	100163	0.5621
0.5539	42.0	102606	0.5535
0.5513	43.0	105049	0.5514
0.5538	44.0	107492	0.5480
0.5423	45.0	109935	0.5488
0.5431	46.0	112378	0.5466
0.5495	47.0	114821	0.5442
0.5593	48.0	117264	0.5447
0.5488	49.0	119707	0.5431
0.5203	50.0	122150	0.5391
0.5386	51.0	124593	0.5384
0.5498	52.0	127036	0.5393
0.5391	53.0	129479	0.5372
0.5361	54.0	131922	0.5363
0.5295	55.0	134365	0.5343
0.5227	56.0	136808	0.5345
0.5182	57.0	139251	0.5287
0.5103	58.0	141694	0.5303
0.5411	59.0	144137	0.5278
0.5187	60.0	146580	0.5259
0.5272	61.0	149023	0.5254
0.5352	62.0	151466	0.5264
0.5243	63.0	153909	0.5214
0.5134	64.0	156352	0.5210
0.5305	65.0	158795	0.5238
0.5507	66.0	161238	0.5210
0.5179	67.0	163681	0.5217
0.5162	68.0	166124	0.5166
0.5192	69.0	168567	0.5201
0.5231	70.0	171010	0.5175
0.5095	71.0	173453	0.5138
0.5205	72.0	175896	0.5135
0.5299	73.0	178339	0.5147
0.4947	74.0	180782	0.5112
0.5133	75.0	183225	0.5115
0.4886	76.0	185668	0.5090
0.5288	77.0	188111	0.5105
0.514	78.0	190554	0.5072
0.4803	79.0	192997	0.5053
0.4882	80.0	195440	0.5075
0.5037	81.0	197883	0.5063
0.5314	82.0	200326	0.5027
0.5181	83.0	202769	0.5013
0.5191	84.0	205212	0.5009
0.503	85.0	207655	0.4980
0.4894	86.0	210098	0.4993
0.4801	87.0	212541	0.4964
0.5019	88.0	214984	0.4956
0.5036	89.0	217427	0.4927
0.4844	90.0	219870	0.4932
0.4656	91.0	222313	0.4890
0.4839	92.0	224756	0.4881
0.4955	93.0	227199	0.4880
0.4792	94.0	229642	0.4877
0.4655	95.0	232085	0.4833
0.4811	96.0	234528	0.4835
0.5118	97.0	236971	0.4842
0.479	98.0	239414	0.4830
0.4693	99.0	241857	0.4827
0.46	100.0	244300	0.4785
0.479	101.0	246743	0.4792
0.4702	102.0	249186	0.4793
0.4683	103.0	251629	0.4757
0.4682	104.0	254072	0.4750
0.4749	105.0	256515	0.4747
0.4915	106.0	258958	0.4719
0.4832	107.0	261401	0.4729
0.4371	108.0	263844	0.4720
0.4779	109.0	266287	0.4710
0.4796	110.0	268730	0.4693
0.463	111.0	271173	0.4696
0.4722	112.0	273616	0.4679
0.4689	113.0	276059	0.4693
0.4644	114.0	278502	0.4665
0.4688	115.0	280945	0.4674
0.4619	116.0	283388	0.4644
0.4533	117.0	285831	0.4663
0.4604	118.0	288274	0.4634
0.4722	119.0	290717	0.4637
0.4622	120.0	293160	0.4634
0.4575	121.0	295603	0.4628
0.4824	122.0	298046	0.4631
0.4757	123.0	300489	0.4620
0.4457	124.0	302932	0.4620
0.4471	125.0	305375	0.4599
0.444	126.0	307818	0.4575
0.4521	127.0	310261	0.4599
0.4441	128.0	312704	0.4588
0.4432	129.0	315147	0.4596
0.4518	130.0	317590	0.4550
0.4457	131.0	320033	0.4578
0.4529	132.0	322476	0.4543
0.4871	133.0	324919	0.4560
0.4482	134.0	327362	0.4546
0.4648	135.0	329805	0.4574
0.4372	136.0	332248	0.4546
0.4353	137.0	334691	0.4531
0.4446	138.0	337134	0.4539
0.4666	139.0	339577	0.4518
0.4734	140.0	342020	0.4528
0.4601	141.0	344463	0.4540
0.4415	142.0	346906	0.4528
0.459	143.0	349349	0.4505
0.454	144.0	351792	0.4514
0.4606	145.0	354235	0.4511
0.4315	146.0	356678	0.4514
0.4583	147.0	359121	0.4520
0.452	148.0	361564	0.4495
0.4449	149.0	364007	0.4508
0.4272	150.0	366450	0.4489
0.439	151.0	368893	0.4504
0.4586	152.0	371336	0.4503
0.4559	153.0	373779	0.4500
0.4527	154.0	376222	0.4492
0.4511	155.0	378665	0.4491
0.4405	156.0	381108	0.4488
0.4509	157.0	383551	0.4482
0.4713	158.0	385994	0.4480
0.4578	159.0	388437	0.4465
0.4154	160.0	390880	0.4464
0.4399	161.0	393323	0.4488
0.4547	162.0	395766	0.4476
0.4426	163.0	398209	0.4456
0.4517	164.0	400652	0.4484
0.4376	165.0	403095	0.4455
0.4463	166.0	405538	0.4463
0.4289	167.0	407981	0.4466
0.4291	168.0	410424	0.4469
0.4623	169.0	412867	0.4455
0.4673	170.0	415310	0.4455
0.4609	171.0	417753	0.4456
0.4478	172.0	420196	0.4468
0.4521	173.0	422639	0.4437
0.4378	174.0	425082	0.4460
0.4361	175.0	427525	0.4446
0.4321	176.0	429968	0.4451
0.4369	177.0	432411	0.4451
0.4381	178.0	434854	0.4443
0.4408	179.0	437297	0.4449
0.4414	180.0	439740	0.4448
0.4333	181.0	442183	0.4438
0.4468	182.0	444626	0.4452
0.4394	183.0	447069	0.4440
0.441	184.0	449512	0.4434
0.4546	185.0	451955	0.4462
0.4455	186.0	454398	0.4458
0.4431	187.0	456841	0.4426
0.4489	188.0	459284	0.4433
0.4485	189.0	461727	0.4435
0.4449	190.0	464170	0.4433
0.4482	191.0	466613	0.4449
0.4395	192.0	469056	0.4433
0.4557	193.0	471499	0.4436
0.4208	194.0	473942	0.4450
0.4274	195.0	476385	0.4429
0.4423	196.0	478828	0.4434
0.4331	197.0	481271	0.4453
0.43	198.0	483714	0.4448
0.4308	199.0	486157	0.4460
0.4373	200.0	488600	0.4430

Framework versions

• Transformers 4.39.0.dev0
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2