yip-i/torgo_xlsr_finrtune-F03-no-lm

torgo_xlsr_finetune-F03-2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5454
• Wer: 0.8555

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
25.1806	0.97	500	3.3512	1.0
3.3648	1.94	1000	3.1566	1.0
2.9865	2.91	1500	2.8249	1.0
2.8219	3.88	2000	2.7880	1.0
2.62	4.85	2500	2.4134	1.1793
2.0129	5.83	3000	1.7735	1.3777
1.3439	6.8	3500	1.4148	1.3656
0.9587	7.77	4000	1.3914	1.2437
0.7532	8.74	4500	1.2565	1.2957
0.6204	9.71	5000	1.2621	1.1074
0.5367	10.68	5500	1.3255	1.1199
0.4471	11.65	6000	1.2730	1.0789
0.3989	12.62	6500	1.2627	1.0258
0.3562	13.59	7000	1.3006	0.9754
0.3346	14.56	7500	1.2739	0.9598
0.2949	15.53	8000	1.3260	0.9238
0.2816	16.5	8500	1.3446	0.9152
0.2552	17.48	9000	1.3537	0.8848
0.2434	18.45	9500	1.3288	0.9258
0.2156	19.42	10000	1.3863	0.8812
0.2126	20.39	10500	1.3466	0.8867
0.1939	21.36	11000	1.4522	0.9113
0.1829	22.33	11500	1.5253	0.8922
0.179	23.3	12000	1.4589	0.8543
0.1684	24.27	12500	1.5436	0.8664
0.1516	25.24	13000	1.5324	0.8668
0.1472	26.21	13500	1.5561	0.8711
0.1399	27.18	14000	1.5400	0.8605
0.1405	28.16	14500	1.5626	0.8512
0.1349	29.13	15000	1.5454	0.8555

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 1.18.3
• Tokenizers 0.13.2