wav2vec2-demo-M02-2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2709
• Wer: 1.0860

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
23.4917	0.91	500	3.2945	1.0
3.4102	1.81	1000	3.1814	1.0
2.9438	2.72	1500	2.7858	1.0
2.6698	3.62	2000	2.4745	1.0035
1.9542	4.53	2500	1.8675	1.3745
1.2737	5.43	3000	1.6459	1.3703
0.9748	6.34	3500	1.8406	1.3037
0.7696	7.25	4000	1.5086	1.2476
0.6396	8.15	4500	1.8280	1.2476
0.558	9.06	5000	1.7680	1.2247
0.4865	9.96	5500	1.8210	1.2309
0.4244	10.87	6000	1.7910	1.1775
0.3898	11.78	6500	1.8021	1.1831
0.3456	12.68	7000	1.7746	1.1456
0.3349	13.59	7500	1.8969	1.1519
0.3233	14.49	8000	1.7402	1.1234
0.3046	15.4	8500	1.8585	1.1429
0.2622	16.3	9000	1.6687	1.0950
0.2593	17.21	9500	1.8192	1.1144
0.2541	18.12	10000	1.8665	1.1110
0.2098	19.02	10500	1.9996	1.1186
0.2192	19.93	11000	2.0346	1.1040
0.1934	20.83	11500	2.1924	1.1012
0.2034	21.74	12000	1.8060	1.0929
0.1857	22.64	12500	2.0334	1.0798
0.1819	23.55	13000	2.1223	1.1040
0.1621	24.46	13500	2.1795	1.0957
0.1548	25.36	14000	2.1545	1.1089
0.1512	26.27	14500	2.2707	1.1186
0.1472	27.17	15000	2.1698	1.0888
0.1296	28.08	15500	2.2496	1.0867
0.1312	28.99	16000	2.2969	1.0881
0.1331	29.89	16500	2.2709	1.0860

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 1.18.3
• Tokenizers 0.13.2