torgo_xlsr_finetune-M03-2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2376
• Wer: 0.5541

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
23.6177	0.92	500	3.4289	1.0
3.4209	1.85	1000	21.5765	1.0
3.1586	2.77	1500	2.8397	1.0
2.7993	3.69	2000	2.7192	1.2867
2.644	4.61	2500	2.5331	1.2996
2.4662	5.54	3000	2.1750	1.2341
1.9879	6.46	3500	1.3732	1.2693
1.4941	7.38	4000	0.8590	1.1900
1.1848	8.3	4500	0.6774	1.1339
0.9662	9.23	5000	0.5184	0.9856
0.8094	10.15	5500	0.4515	0.9504
0.6835	11.07	6000	0.3616	0.8457
0.6111	11.99	6500	0.3209	0.8254
0.5305	12.92	7000	0.3098	0.7902
0.479	13.84	7500	0.2964	0.7569
0.4369	14.76	8000	0.2447	0.7063
0.3836	15.68	8500	0.2676	0.7063
0.3628	16.61	9000	0.2714	0.7128
0.3416	17.53	9500	0.2664	0.6766
0.3297	18.45	10000	0.2510	0.6528
0.2883	19.37	10500	0.2636	0.6493
0.2694	20.3	11000	0.2556	0.6255
0.2655	21.22	11500	0.2328	0.6186
0.2364	22.14	12000	0.2293	0.6037
0.241	23.06	12500	0.2587	0.5928
0.2125	23.99	13000	0.2528	0.5843
0.2101	24.91	13500	0.2315	0.5719
0.1973	25.83	14000	0.2401	0.5769
0.1914	26.75	14500	0.2380	0.5610
0.1936	27.68	15000	0.2425	0.5551
0.1808	28.6	15500	0.2425	0.5556
0.1739	29.52	16000	0.2376	0.5541

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 1.18.3
• Tokenizers 0.13.2