w2v2-base-pretrained_lr5e-5_at0.6_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4078
• Wer: 0.1662

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
25.5608	4.24	250	20.1943	1.0
5.6218	8.47	500	3.4933	1.0
3.134	12.71	750	3.1332	1.0
3.0842	16.95	1000	3.1196	1.0
2.6714	21.19	1250	1.4095	0.9932
0.5642	25.42	1500	0.6028	0.4327
0.2724	29.66	1750	0.7355	0.2396
0.1676	33.9	2000	0.9693	0.2264
0.1247	38.14	2250	1.0615	0.2008
0.0945	42.37	2500	1.2612	0.1841
0.0784	46.61	2750	1.1492	0.1850
0.0672	50.85	3000	1.3113	0.1751
0.0567	55.08	3250	1.3594	0.1721
0.0507	59.32	3500	1.4195	0.1721
0.0478	63.56	3750	1.3916	0.1670
0.0445	67.8	4000	1.4078	0.1662

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1