ASR-model-20k-04

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5806
• Wer: 0.2835

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 12
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.6676	0.4398	500	3.3943	0.9999
3.3831	0.8795	1000	inf	0.9999
2.1618	1.3193	1500	inf	0.7336
1.3088	1.7590	2000	inf	0.5458
1.0799	2.1988	2500	0.8348	0.4859
0.9799	2.6385	3000	0.7635	0.4380
0.915	3.0783	3500	inf	0.4093
0.8476	3.5180	4000	0.7075	0.3845
0.8279	3.9578	4500	inf	0.3703
0.785	4.3975	5000	0.6689	0.3606
0.7549	4.8373	5500	0.6480	0.3461
0.7082	5.2770	6000	0.6163	0.3336
0.7017	5.7168	6500	0.6111	0.3276
0.6665	6.1566	7000	0.6250	0.3203
0.6561	6.5963	7500	0.6099	0.3120
0.651	7.0361	8000	0.6173	0.3079
0.6202	7.4758	8500	inf	0.3045
0.6114	7.9156	9000	inf	0.3001
0.5949	8.3553	9500	0.5861	0.2961
0.5933	8.7951	10000	inf	0.2948
0.5757	9.2348	10500	0.5854	0.2923
0.5632	9.6746	11000	inf	0.2921
0.5504	10.1143	11500	inf	0.2891
0.544	10.5541	12000	inf	0.2874
0.544	10.9938	12500	inf	0.2844
0.5323	11.4336	13000	inf	0.2841
0.5331	11.8734	13500	0.5806	0.2835

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 2.14.4
• Tokenizers 0.21.1