lr-1e4-model

This model is a fine-tuned version of anvitamanne/base-model on the None dataset. It achieves the following results on the evaluation set:

• Loss: 577.1107
• Wer: 0.3916
• Cer: 0.1674

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
315.5201	0.86	1000	498.0310	0.4048	0.1649
324.3248	1.72	2000	522.2733	0.4075	0.1662
313.0174	2.58	3000	515.3785	0.4113	0.1698
303.8601	3.44	4000	507.4600	0.3983	0.1625
289.2461	4.3	5000	493.6467	0.4135	0.1672
285.9346	5.17	6000	527.8949	0.4027	0.1665
279.1058	6.03	7000	540.5751	0.3958	0.1647
279.2655	6.89	8000	520.2695	0.3979	0.1664
279.1603	7.75	9000	550.3109	0.4029	0.1693
261.9772	8.61	10000	535.4487	0.3958	0.1660
258.0145	9.47	11000	556.1581	0.3974	0.1649
246.9439	10.33	12000	570.6581	0.3954	0.1673
269.0996	11.19	13000	595.9214	0.3871	0.1639
252.4734	12.05	14000	579.9908	0.3900	0.1655
248.6116	12.91	15000	581.7884	0.3939	0.1675
253.1624	13.78	16000	582.6721	0.3915	0.1668
261.0992	14.64	17000	577.1107	0.3916	0.1674

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.2+cu118
• Datasets 3.6.0
• Tokenizers 0.15.2