fineturning-without-pretraining

This model is a fine-tuned version of on the common_voice_1_0 dataset. It achieves the following results on the evaluation set:

• Loss: 812.3214
• Wer: 1.1838

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2141.1597	2.15	500	908.3237	1.0
1549.364	4.29	1000	813.9642	1.1682
1442.1247	6.44	1500	788.9472	1.5341
1395.3347	8.58	2000	757.5609	1.2662
1344.5591	10.73	2500	751.7140	1.1790
1289.6164	12.88	3000	746.7259	1.2651
1248.0024	15.02	3500	761.1828	1.2634
1208.4588	17.17	4000	789.4526	1.1426
1162.758	19.31	4500	794.2302	1.1521
1118.2571	21.46	5000	803.2517	1.2117
1097.6801	23.61	5500	812.3214	1.1838

Framework versions

• Transformers 4.39.3
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2