Edit model card

wav2vec2-large-xls-r-53h-turkish-colab

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the common_voice dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4135
  • Wer: 0.3247

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 32
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
9.4875 0.92 100 3.5328 1.0
3.1866 1.83 200 3.0955 1.0
2.027 2.75 300 0.9002 0.7685
0.7285 3.67 400 0.6279 0.6693
0.4693 4.59 500 0.5672 0.5643
0.3615 5.5 600 0.4995 0.5094
0.2846 6.42 700 0.4561 0.4797
0.2253 7.34 800 0.4742 0.4675
0.2004 8.26 900 0.4462 0.4345
0.173 9.17 1000 0.4688 0.4333
0.1547 10.09 1100 0.4429 0.4206
0.1444 11.01 1200 0.4662 0.4144
0.1274 11.93 1300 0.4675 0.4213
0.1164 12.84 1400 0.4947 0.4073
0.1081 13.76 1500 0.4223 0.3915
0.1025 14.68 1600 0.4493 0.3912
0.0944 15.6 1700 0.4527 0.3848
0.0943 16.51 1800 0.4288 0.3810
0.0885 17.43 1900 0.4313 0.3670
0.0781 18.35 2000 0.4729 0.3790
0.0828 19.27 2100 0.4560 0.3651
0.0753 20.18 2200 0.4478 0.3599
0.0702 21.1 2300 0.4518 0.3595
0.0666 22.02 2400 0.4080 0.3489
0.0661 22.94 2500 0.4414 0.3507
0.0607 23.85 2600 0.4209 0.3538
0.058 24.77 2700 0.4302 0.3382
0.0596 25.69 2800 0.3939 0.3328
0.052 26.61 2900 0.4374 0.3311
0.0473 27.52 3000 0.4406 0.3363
0.0483 28.44 3100 0.4272 0.3286
0.049 29.36 3200 0.4189 0.3257
0.0433 30.28 3300 0.4242 0.3229
0.0438 31.19 3400 0.4135 0.3247

Framework versions

  • Transformers 4.11.3
  • Pytorch 1.10.0+cu113
  • Datasets 1.18.3
  • Tokenizers 0.10.3
Downloads last month
2

Dataset used to train hamidov02/wav2vec2-large-xls-r-53h-turkish-colab