w2v2-base-pretrained_lr5e-5_at0.8_da0.4

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.6336
• Wer: 0.8377

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
27.1555	13.16	250	16.9359	1.0
8.9276	26.32	500	9.0668	1.0
6.6067	39.47	750	8.0040	1.0
3.8775	52.63	1000	5.5239	1.0
3.1858	65.79	1250	5.5803	1.0
3.0238	78.95	1500	5.5821	1.0004
2.7704	92.11	1750	5.6283	1.0068
2.4867	105.26	2000	5.4805	0.9855
2.2251	118.42	2250	5.6338	0.9645
1.9731	131.58	2500	5.5967	0.9197
1.752	144.74	2750	5.5503	0.8949
1.5403	157.89	3000	5.4723	0.8663
1.3664	171.05	3250	5.5707	0.8603
1.2337	184.21	3500	5.5728	0.8449
1.1468	197.37	3750	5.6135	0.8402
1.0931	210.53	4000	5.6336	0.8377

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1