wav2vec2-turkish-300m-3

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice_16_1 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2968
• Wer: 0.2453

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.015	0.3652	500	0.5674	0.5880
0.5687	0.7305	1000	0.4913	0.5483
0.4727	1.0957	1500	0.4382	0.4868
0.3713	1.4609	2000	0.3941	0.4761
0.3653	1.8262	2500	0.3978	0.4609
0.3457	2.1914	3000	0.3570	0.4201
0.3108	2.5566	3500	0.3273	0.4045
0.2985	2.9218	4000	0.3559	0.4253
0.2768	3.2871	4500	0.3484	0.4288
0.2702	3.6523	5000	0.3422	0.3988
0.2626	4.0175	5500	0.3312	0.3875
0.246	4.3828	6000	0.3175	0.3735
0.2373	4.7480	6500	0.3126	0.3750
0.234	5.1132	7000	0.3289	0.3703
0.2225	5.4785	7500	0.3170	0.3700
0.2094	5.8437	8000	0.3127	0.3611
0.1961	6.2089	8500	0.3130	0.3604
0.1927	6.5741	9000	0.3167	0.3491
0.1963	6.9394	9500	0.2983	0.3451
0.1757	7.3046	10000	0.3044	0.3403
0.1732	7.6698	10500	0.2988	0.3407
0.1737	8.0351	11000	0.3128	0.3367
0.1686	8.4003	11500	0.2954	0.3296
0.1588	8.7655	12000	0.3226	0.3265
0.1481	9.1308	12500	0.2946	0.3172
0.1434	9.4960	13000	0.2981	0.3202
0.146	9.8612	13500	0.2936	0.3150
0.1352	10.2264	14000	0.2895	0.3091
0.1304	10.5917	14500	0.2932	0.3071
0.1253	10.9569	15000	0.2946	0.2997
0.12	11.3221	15500	0.2967	0.3065
0.1179	11.6874	16000	0.2856	0.3037
0.1185	12.0526	16500	0.2753	0.2973
0.1128	12.4178	17000	0.2954	0.2935
0.1054	12.7831	17500	0.2917	0.2916
0.1026	13.1483	18000	0.2878	0.2820
0.0981	13.5135	18500	0.2882	0.2863
0.0936	13.8787	19000	0.2758	0.2774
0.0911	14.2440	19500	0.2867	0.2811
0.0881	14.6092	20000	0.2952	0.2760
0.0809	14.9744	20500	0.2996	0.2772
0.0815	15.3397	21000	0.2806	0.2694
0.078	15.7049	21500	0.3050	0.2717
0.0727	16.0701	22000	0.2871	0.2682
0.0716	16.4354	22500	0.2935	0.2667
0.0672	16.8006	23000	0.2917	0.2632
0.0666	17.1658	23500	0.3075	0.2584
0.0654	17.5310	24000	0.3025	0.2580
0.0616	17.8963	24500	0.2952	0.2550
0.0609	18.2615	25000	0.3077	0.2567
0.0604	18.6267	25500	0.3040	0.2513
0.0549	18.9920	26000	0.3043	0.2481
0.0516	19.3572	26500	0.3036	0.2476
0.0543	19.7224	27000	0.2968	0.2453

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.2+cu121
• Datasets 2.17.1
• Tokenizers 0.19.1