ep-30-model

This model is a fine-tuned version of anvitamanne/base-model on the None dataset. It achieves the following results on the evaluation set:

• Loss: 603.7294
• Wer: 0.3891
• Cer: 0.1674

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
313.4229	0.86	1000	512.8486	0.4031	0.1662
312.2834	1.72	2000	509.6784	0.3964	0.1643
303.4682	2.58	3000	521.8994	0.3944	0.1642
299.0293	3.44	4000	489.4095	0.3982	0.1629
286.2679	4.3	5000	516.8929	0.4048	0.1660
285.5344	5.17	6000	550.6877	0.4034	0.1672
278.8618	6.03	7000	549.6069	0.4035	0.1671
281.2304	6.89	8000	536.3907	0.3991	0.1653
281.8211	7.75	9000	569.9989	0.4124	0.1700
266.6356	8.61	10000	531.8161	0.4015	0.1670
263.5382	9.47	11000	573.9767	0.4035	0.1683
253.7602	10.33	12000	566.3726	0.4052	0.1695
276.6175	11.19	13000	576.7356	0.4027	0.1693
260.0645	12.05	14000	573.5627	0.3988	0.1665
257.4325	12.91	15000	569.2803	0.4014	0.1684
263.3572	13.78	16000	574.4833	0.4014	0.1680
271.3235	14.64	17000	568.9285	0.3937	0.1645
271.2437	15.5	18000	560.3303	0.3950	0.1660
272.6667	16.36	19000	559.9153	0.3968	0.1670
268.6009	17.22	20000	566.6968	0.3959	0.1666
274.8418	18.08	21000	578.3120	0.3931	0.1659
268.7353	18.94	22000	560.3764	0.3973	0.1675
253.8548	19.8	23000	572.3874	0.3913	0.1654
263.4848	20.66	24000	584.7192	0.3919	0.1655
261.7505	21.52	25000	585.3862	0.3948	0.1671
264.9873	22.38	26000	591.625	0.3908	0.1660
261.2484	23.25	27000	586.8426	0.3907	0.1670
261.3986	24.11	28000	598.3438	0.3882	0.1661
250.799	24.97	29000	593.3273	0.3905	0.1672
247.0973	25.83	30000	600.5747	0.3880	0.1669
253.7963	26.69	31000	605.4449	0.3899	0.1673
254.9214	27.55	32000	604.3179	0.3916	0.1674
248.1459	28.41	33000	605.5740	0.3914	0.1671
255.9482	29.27	34000	603.7294	0.3891	0.1674

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu118
• Datasets 3.6.0
• Tokenizers 0.15.2