model_weight_with_token_110_11

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0692
• Cer: 0.0582
• Wer: 0.1738

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Wer
0.5359	1.3928	500	0.1281	0.0733	0.2101
0.4994	2.7855	1000	0.1591	0.0731	0.2098
0.4735	4.1783	1500	0.1169	0.0708	0.2022
0.4432	5.5710	2000	0.1102	0.0710	0.2031
0.4204	6.9638	2500	0.1127	0.0735	0.2023
0.3919	8.3565	3000	0.1146	0.0699	0.1992
0.3701	9.7493	3500	0.1047	0.0712	0.1993
0.3826	11.1421	4000	0.1073	0.0688	0.1952
0.3512	12.5348	4500	0.0898	0.0650	0.1894
0.327	13.9276	5000	0.0931	0.0650	0.1869
0.3284	15.3203	5500	0.1028	0.0672	0.1918
0.3016	16.7131	6000	0.0973	0.0651	0.1875
0.2893	18.1058	6500	0.0899	0.0638	0.1849
0.292	19.4986	7000	0.0882	0.0632	0.1840
0.2688	20.8914	7500	0.0808	0.0625	0.1820
0.2672	22.2841	8000	0.0807	0.0620	0.1818
0.266	23.6769	8500	0.0807	0.0621	0.1816
0.2499	25.0696	9000	0.0741	0.0615	0.1787
0.244	26.4624	9500	0.0780	0.0620	0.1803
0.2403	27.8552	10000	0.0764	0.0607	0.1791
0.2268	29.2479	10500	0.0750	0.0599	0.1778
0.2318	30.6407	11000	0.0728	0.0600	0.1775
0.2143	32.0334	11500	0.0723	0.0592	0.1757
0.2193	33.4262	12000	0.0699	0.0591	0.1749
0.1992	34.8189	12500	0.0717	0.0585	0.1748
0.2238	36.2117	13000	0.0692	0.0587	0.1744
0.2074	37.6045	13500	0.0696	0.0581	0.1735
0.1984	38.9972	14000	0.0692	0.0582	0.1738

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1