Edit model card

wav2vec2-19

WER 0.283

WER 0.126 with 3-Gram

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6305
  • Wer: 0.4499

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 800
  • num_epochs: 60

Training results

Training Loss Epoch Step Validation Loss Wer
3.4816 2.74 400 1.0717 0.8927
0.751 5.48 800 0.7155 0.7533
0.517 8.22 1200 0.7039 0.6675
0.3988 10.96 1600 0.5935 0.6149
0.3179 13.7 2000 0.6477 0.5999
0.2755 16.44 2400 0.5549 0.5798
0.2343 19.18 2800 0.6626 0.5798
0.2103 21.92 3200 0.6488 0.5674
0.1877 24.66 3600 0.5874 0.5339
0.1719 27.4 4000 0.6354 0.5389
0.1603 30.14 4400 0.6612 0.5210
0.1401 32.88 4800 0.6676 0.5131
0.1286 35.62 5200 0.6366 0.5075
0.1159 38.36 5600 0.6064 0.4977
0.1084 41.1 6000 0.6530 0.4835
0.0974 43.84 6400 0.6118 0.4853
0.0879 46.58 6800 0.6316 0.4770
0.0815 49.32 7200 0.6125 0.4664
0.0708 52.05 7600 0.6449 0.4683
0.0651 54.79 8000 0.6068 0.4571
0.0555 57.53 8400 0.6305 0.4499

Framework versions

  • Transformers 4.19.2
  • Pytorch 1.11.0+cu113
  • Datasets 2.2.2
  • Tokenizers 0.12.1
Downloads last month
5