wav2vec2-burak-new-v10-small

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3345
• Wer: 0.2030

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 271
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.1239	9.43	500	3.1263	1.0
1.7776	18.87	1000	0.3793	0.4838
0.5275	28.3	1500	0.2654	0.3379
0.3605	37.74	2000	0.2704	0.2953
0.2802	47.17	2500	0.2610	0.2911
0.2348	56.6	3000	0.2717	0.2677
0.2101	66.04	3500	0.2736	0.2691
0.1805	75.47	4000	0.2782	0.2595
0.1644	84.91	4500	0.2873	0.2491
0.1469	94.34	5000	0.3040	0.2381
0.138	103.77	5500	0.3205	0.2429
0.1247	113.21	6000	0.3217	0.2264
0.118	122.64	6500	0.3148	0.2244
0.1116	132.08	7000	0.3114	0.2209
0.1045	141.51	7500	0.3151	0.2175
0.0988	150.94	8000	0.3096	0.2092
0.0925	160.38	8500	0.3357	0.2230
0.0898	169.81	9000	0.3220	0.2099
0.0848	179.25	9500	0.3372	0.2209
0.0831	188.68	10000	0.3030	0.2030
0.0796	198.11	10500	0.3297	0.2127
0.0747	207.55	11000	0.3312	0.2134
0.0777	216.98	11500	0.3231	0.2168
0.0724	226.42	12000	0.3248	0.2078
0.0705	235.85	12500	0.3277	0.2023
0.0691	245.28	13000	0.3262	0.1996
0.0661	254.72	13500	0.3356	0.1996
0.0678	264.15	14000	0.3345	0.2030

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu116
• Datasets 2.7.1
• Tokenizers 0.13.2