w2v2-base-pretrained_lr5e-5_at0.9_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.5021
• Wer: 0.1820

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
18.4753	6.1	250	4.2333	1.0
3.358	12.2	500	3.2161	1.0
3.108	18.29	750	3.1259	1.0
2.2419	24.39	1000	1.1771	0.7117
0.2791	30.49	1250	1.5708	0.2328
0.137	36.59	1500	1.7979	0.1961
0.0936	42.68	1750	2.0223	0.1948
0.0723	48.78	2000	2.2080	0.1892
0.0584	54.88	2250	2.1748	0.1854
0.0472	60.98	2500	2.2649	0.1828
0.042	67.07	2750	2.3289	0.1880
0.0349	73.17	3000	2.3042	0.1854
0.032	79.27	3250	2.2214	0.1841
0.0284	85.37	3500	2.3253	0.1807
0.0258	91.46	3750	2.4980	0.1815
0.0238	97.56	4000	2.5021	0.1820

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1