wav2vec2-large-xlsr-53-torgo-demo-f01-nolm

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0153
• Wer: 0.4756

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.4166	0.81	500	4.5019	1.0
3.1088	1.62	1000	3.0459	1.0
2.8249	2.44	1500	3.0850	1.0
2.625	3.25	2000	2.6827	1.3656
1.9816	4.06	2500	1.6636	1.3701
1.3036	4.87	3000	0.9710	1.2504
0.9862	5.68	3500	0.6023	1.0519
0.7012	6.49	4000	0.4404	0.9342
0.6102	7.31	4500	0.3297	0.8491
0.5463	8.12	5000	0.2403	0.7773
0.4897	8.93	5500	0.1907	0.7335
0.4687	9.74	6000	0.1721	0.7095
0.41	10.55	6500	0.1382	0.6851
0.3277	11.36	7000	0.1189	0.6598
0.3182	12.18	7500	0.1040	0.6372
0.3279	12.99	8000	0.0961	0.6274
0.2735	13.8	8500	0.0806	0.5880
0.3153	14.61	9000	0.0821	0.5748
0.251	15.42	9500	0.0633	0.5437
0.2	16.23	10000	0.0534	0.5316
0.2134	17.05	10500	0.0475	0.5195
0.1727	17.86	11000	0.0435	0.5146
0.2143	18.67	11500	0.0406	0.5072
0.1679	19.48	12000	0.0386	0.5057
0.1836	20.29	12500	0.0359	0.4984
0.1542	21.1	13000	0.0284	0.4914
0.1672	21.92	13500	0.0289	0.4884
0.1526	22.73	14000	0.0256	0.4867
0.1263	23.54	14500	0.0247	0.4871
0.133	24.35	15000	0.0194	0.4816
0.1005	25.16	15500	0.0190	0.4798
0.1372	25.97	16000	0.0172	0.4786
0.1126	26.79	16500	0.0177	0.4773
0.0929	27.6	17000	0.0173	0.4775
0.1069	28.41	17500	0.0164	0.4773
0.0932	29.22	18000	0.0153	0.4756

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.0.0
• Tokenizers 0.13.2