Edit model card

wav2vec2-large-xls-r-300m-test1yakutsi-colab

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4509
  • Wer: 0.4245

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 250
  • num_epochs: 10
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
3.1089 1.1707 120 2.9271 1.0
2.1217 2.3415 240 0.8076 0.7261
0.5442 3.5122 360 0.4935 0.5490
0.3041 4.6829 480 0.4464 0.4832
0.2184 5.8537 600 0.4263 0.4554
0.1675 7.0244 720 0.4416 0.4488
0.138 8.1951 840 0.4512 0.4380
0.1167 9.3659 960 0.4509 0.4245

Framework versions

  • Transformers 4.41.0.dev0
  • Pytorch 2.2.1+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
8
Safetensors
Model size
315M params
Tensor type
F32
·

Finetuned from

Evaluation results