Edit model card

wav2vec2-large-xls-r-300m-russian-colab-beam_search_test

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7619
  • Wer: 0.4680

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 800
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
10.0158 4.16 100 5.4134 1.0
4.0394 8.33 200 3.4304 1.0
3.2721 12.49 300 3.2273 1.0
3.1277 16.66 400 2.8023 0.9984
1.3791 20.82 500 0.9888 0.8546
0.3659 24.99 600 0.7602 0.6304
0.1858 29.16 700 0.7965 0.6156
0.1403 33.33 800 0.7998 0.5839
0.1173 37.49 900 0.8353 0.5941
0.0917 41.66 1000 0.8272 0.5522
0.0743 45.82 1100 0.8342 0.5471
0.063 49.99 1200 0.7988 0.5352
0.0528 54.16 1300 0.7740 0.5201
0.0456 58.33 1400 0.7636 0.5165
0.0389 62.49 1500 0.7922 0.5161
0.0329 66.66 1600 0.8035 0.5158
0.0283 70.82 1700 0.7873 0.4832
0.0255 74.99 1800 0.7853 0.4870
0.0236 79.16 1900 0.8236 0.5045
0.0202 83.33 2000 0.7661 0.4796
0.0165 87.49 2100 0.7584 0.4680
0.0156 91.66 2200 0.7685 0.4772
0.0149 95.82 2300 0.7519 0.4696
0.0126 99.99 2400 0.7619 0.4680

Framework versions

  • Transformers 4.11.3
  • Pytorch 1.10.0+cu111
  • Datasets 1.18.3
  • Tokenizers 0.10.3
Downloads last month
12
Hosted inference API
or or
This model can be loaded on the Inference API on-demand.

Dataset used to train jfealko/wav2vec2-large-xls-r-300m-russian-colab-beam_search_test