xls-r-300-cv17-polish-adap-ru

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4087
• Wer: 0.2986
• Cer: 0.0652

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.2673	1.6	100	3.3121	1.0	1.0
1.2344	3.2	200	1.1417	0.8846	0.2502
0.4279	4.8	300	0.4485	0.4848	0.1082
0.2415	6.4	400	0.3752	0.3971	0.0871
0.2634	8.0	500	0.4058	0.4148	0.0927
0.1683	9.6	600	0.4079	0.3906	0.0887
0.1356	11.2	700	0.4017	0.3927	0.0872
0.0887	12.8	800	0.4094	0.3867	0.0874
0.1529	14.4	900	0.4055	0.3728	0.0843
0.1206	16.0	1000	0.4030	0.3709	0.0824
0.0573	17.6	1100	0.4370	0.3787	0.0841
0.073	19.2	1200	0.4157	0.3653	0.0819
0.0498	20.8	1300	0.4235	0.3637	0.0811
0.0987	22.4	1400	0.4153	0.3526	0.0786
0.0791	24.0	1500	0.4239	0.3557	0.0802
0.0698	25.6	1600	0.4253	0.3473	0.0779
0.0745	27.2	1700	0.4092	0.3518	0.0784
0.0689	28.8	1800	0.4326	0.3433	0.0764
0.059	30.4	1900	0.4207	0.3342	0.0738
0.0255	32.0	2000	0.4053	0.3272	0.0726
0.0403	33.6	2100	0.4267	0.3264	0.0715
0.0281	35.2	2200	0.4141	0.3250	0.0719
0.0533	36.8	2300	0.4242	0.3252	0.0718
0.0503	38.4	2400	0.4062	0.3147	0.0690
0.0292	40.0	2500	0.4109	0.3081	0.0676
0.0276	41.6	2600	0.3919	0.3044	0.0665
0.0177	43.2	2700	0.4104	0.3038	0.0664
0.0268	44.8	2800	0.4149	0.3040	0.0662
0.0388	46.4	2900	0.4090	0.3003	0.0656
0.0193	48.0	3000	0.4092	0.2994	0.0652
0.0428	49.6	3100	0.4087	0.2986	0.0652

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1