Edit model card

wav2vec2-large-xls-r-300m-irish-colab_test

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice dataset. It achieves the following results on the evaluation set:

  • Loss: 1.7839
  • Wer: 0.6220

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 90
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
10.0428 2.94 50 4.1311 1.0
3.2917 5.88 100 3.1468 1.0
3.0221 8.82 150 2.9848 1.0
2.9795 11.76 200 2.9567 1.0
2.9379 14.71 250 2.9463 1.0
2.9068 17.65 300 2.8330 1.0
2.5088 20.59 350 1.9807 0.9535
1.6188 23.53 400 1.4254 0.8398
1.0435 26.47 450 1.3668 0.7807
0.7212 29.41 500 1.3914 0.7476
0.5456 32.35 550 1.5495 0.7470
0.4297 35.29 600 1.4751 0.6960
0.3533 38.24 650 1.5157 0.6909
0.2899 41.18 700 1.5394 0.6879
0.2529 44.12 750 1.6186 0.6903
0.2413 47.06 800 1.6386 0.6954
0.2113 50.0 850 1.6906 0.6778
0.1769 52.94 900 1.6918 0.6575
0.1622 55.88 950 1.7313 0.6572
0.1564 58.82 1000 1.7701 0.6510
0.1637 61.76 1050 1.6800 0.6444
0.148 64.71 1100 1.7306 0.6477
0.1385 67.65 1150 1.7605 0.6408
0.1264 70.59 1200 1.7534 0.6244
0.1157 73.53 1250 1.7906 0.6381
0.1027 76.47 1300 1.7803 0.6265
0.1061 79.41 1350 1.7617 0.6259
0.0934 82.35 1400 1.7649 0.6253
0.0904 85.29 1450 1.7713 0.6187
0.0911 88.24 1500 1.7839 0.6220

Framework versions

  • Transformers 4.11.3
  • Pytorch 1.10.0+cu111
  • Datasets 1.18.3
  • Tokenizers 0.10.3
Downloads last month
3
Hosted inference API
or or
This model can be loaded on the Inference API on-demand.

Dataset used to train jfealko/wav2vec2-large-xls-r-300m-irish-colab_test