wav2vec2-turkish-300m-7

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the fleurs dataset. It achieves the following results on the evaluation set:

• Loss: 0.3236
• Wer: 0.1668

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 35
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.7291	0.6983	500	1.2114	0.8908
1.1707	1.3966	1000	0.3888	0.4555
0.5042	2.0950	1500	0.2879	0.3270
0.2623	2.7933	2000	0.2653	0.3265
0.2012	3.4916	2500	0.2405	0.2778
0.1817	4.1899	3000	0.2555	0.2704
0.1394	4.8883	3500	0.2452	0.2647
0.1112	5.5866	4000	0.2426	0.2458
0.1047	6.2849	4500	0.2520	0.2634
0.0916	6.9832	5000	0.2417	0.2443
0.0902	7.6816	5500	0.2627	0.2427
0.075	8.3799	6000	0.2551	0.2320
0.0716	9.0782	6500	0.2607	0.2221
0.0661	9.7765	7000	0.2504	0.2338
0.0634	10.4749	7500	0.2552	0.2229
0.0583	11.1732	8000	0.2637	0.2249
0.0537	11.8715	8500	0.2627	0.2122
0.0535	12.5698	9000	0.2654	0.2148
0.0521	13.2682	9500	0.2665	0.2123
0.0491	13.9665	10000	0.2814	0.2176
0.0466	14.6648	10500	0.2785	0.2138
0.0445	15.3631	11000	0.2856	0.2075
0.0415	16.0615	11500	0.2750	0.2076
0.0405	16.7598	12000	0.2743	0.2045
0.0368	17.4581	12500	0.2770	0.2013
0.0374	18.1564	13000	0.2961	0.2043
0.0374	18.8547	13500	0.2851	0.2028
0.0322	19.5531	14000	0.2955	0.1961
0.0317	20.2514	14500	0.3053	0.1998
0.0306	20.9497	15000	0.2988	0.1960
0.0328	21.6480	15500	0.2873	0.1949
0.0299	22.3464	16000	0.3030	0.1921
0.0272	23.0447	16500	0.2902	0.1866
0.0286	23.7430	17000	0.2962	0.1879
0.0288	24.4413	17500	0.3114	0.1871
0.0253	25.1397	18000	0.3203	0.1844
0.0262	25.8380	18500	0.2993	0.1861
0.0238	26.5363	19000	0.3108	0.1812
0.0228	27.2346	19500	0.3143	0.1759
0.0235	27.9330	20000	0.3077	0.1780
0.0227	28.6313	20500	0.3099	0.1739
0.0212	29.3296	21000	0.3144	0.1730
0.0212	30.0279	21500	0.3165	0.1726
0.0211	30.7263	22000	0.3178	0.1708
0.0192	31.4246	22500	0.3172	0.1682
0.0193	32.1229	23000	0.3188	0.1693
0.0195	32.8212	23500	0.3255	0.1661
0.0179	33.5196	24000	0.3248	0.1668
0.0166	34.2179	24500	0.3261	0.1668
0.018	34.9162	25000	0.3236	0.1668

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.2+cu121
• Datasets 2.17.1
• Tokenizers 0.19.1