hd-0.3-model

This model is a fine-tuned version of anvitamanne/base-model on the None dataset. It achieves the following results on the evaluation set:

• Loss: 560.9241
• Wer: 0.4023
• Cer: 0.1685

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
313.894	0.86	1000	508.5718	0.4055	0.1656
315.6504	1.72	2000	526.5672	0.4005	0.1642
304.3114	2.58	3000	525.9501	0.3996	0.1648
296.7249	3.44	4000	497.6855	0.3972	0.1626
282.7711	4.3	5000	512.9740	0.4060	0.1657
282.1519	5.17	6000	525.6339	0.3989	0.1654
275.2861	6.03	7000	555.5438	0.4032	0.1672
277.682	6.89	8000	532.3320	0.3942	0.1642
279.296	7.75	9000	541.7022	0.3982	0.1679
264.0832	8.61	10000	536.3400	0.3967	0.1665
261.8448	9.47	11000	553.1898	0.4014	0.1682
252.598	10.33	12000	554.9163	0.3989	0.1675
274.7766	11.19	13000	574.4638	0.4000	0.1690
259.2969	12.05	14000	566.6737	0.4019	0.1696
257.0598	12.91	15000	567.9193	0.4031	0.1693
263.2721	13.78	16000	563.6974	0.4034	0.1687
274.2213	14.64	17000	560.9241	0.4023	0.1685

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu118
• Datasets 3.6.0
• Tokenizers 0.15.2