wav2vec2-large-xlsr-53-torgo-demo-m01-nolm

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0161
• Wer: 0.4768

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.3987	0.9	500	4.6666	1.0
2.9362	1.8	1000	3.2475	1.0
2.7871	2.7	1500	2.9266	1.0
2.3782	3.6	2000	2.2303	1.1786
1.4728	4.5	2500	1.1054	1.2930
1.0335	5.41	3000	0.6886	1.1068
0.7658	6.31	3500	0.4544	0.9401
0.633	7.21	4000	0.3153	0.8427
0.5852	8.11	4500	0.2553	0.7724
0.4882	9.01	5000	0.2212	0.7450
0.4167	9.91	5500	0.1687	0.7014
0.3897	10.81	6000	0.1433	0.6791
0.382	11.71	6500	0.1249	0.6434
0.3367	12.61	7000	0.1056	0.6198
0.2897	13.51	7500	0.0988	0.6172
0.2457	14.41	8000	0.0885	0.5895
0.243	15.32	8500	0.0759	0.5850
0.2594	16.22	9000	0.0621	0.5615
0.2461	17.12	9500	0.0583	0.5470
0.2307	18.02	10000	0.0501	0.5449
0.2333	18.92	10500	0.0510	0.5332
0.1579	19.82	11000	0.0447	0.5295
0.2097	20.72	11500	0.0392	0.5246
0.1661	21.62	12000	0.0360	0.5075
0.1576	22.52	12500	0.0287	0.5076
0.1452	23.42	13000	0.0253	0.4988
0.1316	24.32	13500	0.0220	0.4863
0.1521	25.23	14000	0.0222	0.4865
0.1319	26.13	14500	0.0187	0.4810
0.1392	27.03	15000	0.0170	0.4809
0.1619	27.93	15500	0.0171	0.4777
0.1374	28.83	16000	0.0164	0.4766
0.1081	29.73	16500	0.0161	0.4768

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.0.0
• Tokenizers 0.13.2