wav2vec2-burak-new-300-v2-7

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3394
• Wer: 0.2306

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 151
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.3306	9.43	500	2.8304	1.0
0.9789	18.87	1000	0.3443	0.4136
0.3622	28.3	1500	0.3110	0.3235
0.2535	37.74	2000	0.3166	0.3014
0.1985	47.17	2500	0.3247	0.2980
0.1673	56.6	3000	0.3082	0.2608
0.1435	66.04	3500	0.3155	0.2615
0.1281	75.47	4000	0.3480	0.2636
0.1164	84.91	4500	0.3459	0.2560
0.1055	94.34	5000	0.3330	0.2505
0.0949	103.77	5500	0.3522	0.2436
0.0891	113.21	6000	0.3511	0.2381
0.0814	122.64	6500	0.3365	0.2347
0.0785	132.08	7000	0.3426	0.2374
0.0722	141.51	7500	0.3483	0.2312
0.0704	150.94	8000	0.3394	0.2306

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1+cu113
• Datasets 2.7.1
• Tokenizers 0.13.2