w2v2-base-pretrained_lr5e-5_at0.5_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6472
• Wer: 0.1786

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
18.9615	4.17	250	4.3427	1.0
3.3745	8.33	500	3.2218	1.0
3.1073	12.5	750	3.1350	1.0
2.1233	16.67	1000	0.8774	0.7915
0.4649	20.83	1250	0.8879	0.3691
0.2435	25.0	1500	0.8832	0.2119
0.1677	29.17	1750	1.2547	0.1871
0.1148	33.33	2000	1.3004	0.1786
0.0904	37.5	2250	1.2741	0.1995
0.0732	41.67	2500	1.4082	0.1747
0.0613	45.83	2750	1.5234	0.1824
0.0536	50.0	3000	1.4060	0.1798
0.046	54.17	3250	1.5669	0.1833
0.0441	58.33	3500	1.5661	0.1807
0.0392	62.5	3750	1.6374	0.1790
0.0384	66.67	4000	1.6472	0.1786

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1