SriDoc/TrOCR-Sin-Handwritten

TrOCR-SIN-DeiT-Handwritten

This model is a fine-tuned version of kavg/TrOCR-SIN-DeiT on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.9839
• Cer: 0.5253

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
0.2915	3.45	100	1.8613	0.6450
0.061	6.9	200	1.8118	0.5707
0.0363	10.34	300	2.3998	0.6420
0.0202	13.79	400	2.4144	0.6353
0.0329	17.24	500	2.4393	0.6577
0.0364	20.69	600	1.9231	0.5679
0.004	24.14	700	2.4344	0.5866
0.0167	27.59	800	3.0998	0.5744
0.0269	31.03	900	2.6785	0.5804
0.0151	34.48	1000	2.2443	0.5916
0.0008	37.93	1100	2.1480	0.5684
0.0067	41.38	1200	2.3553	0.5625
0.0198	44.83	1300	2.1915	0.5492
0.0002	48.28	1400	2.0370	0.5620
0.001	51.72	1500	2.4303	0.6056
0.1666	55.17	1600	2.3324	0.5627
0.0001	58.62	1700	2.8753	0.5582
0.0	62.07	1800	2.5749	0.5355
0.0002	65.52	1900	2.8105	0.5572
0.0	68.97	2000	2.5275	0.5462
0.1231	72.41	2100	2.7452	0.5477
0.0	75.86	2200	2.4278	0.5403
0.0	79.31	2300	3.0099	0.5487
0.0	82.76	2400	3.1290	0.5467
0.0	86.21	2500	2.7705	0.5263
0.0	89.66	2600	2.7828	0.5275
0.0	93.1	2700	3.2488	0.5345
0.0	96.55	2800	3.1309	0.5273
0.0	100.0	2900	2.9839	0.5253

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1