Edit model card

w2v2-base-pretrained_lr5e-5_at0.0_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.2935
  • Wer: 0.1709

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • training_steps: 4000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
22.479 3.91 250 4.5097 1.0
3.5142 7.81 500 3.2115 1.0
3.144 11.72 750 3.1091 1.0
2.5815 15.62 1000 1.2241 0.9991
0.6479 19.53 1250 0.5868 0.3328
0.3255 23.44 1500 0.7123 0.2050
0.2136 27.34 1750 0.8753 0.1854
0.1561 31.25 2000 0.9095 0.1892
0.1195 35.16 2250 1.0824 0.1828
0.0966 39.06 2500 1.0976 0.1756
0.0829 42.97 2750 1.1946 0.1734
0.0724 46.88 3000 1.2161 0.1713
0.0611 50.78 3250 1.2877 0.1739
0.0555 54.69 3500 1.3169 0.1687
0.0537 58.59 3750 1.2744 0.1764
0.0481 62.5 4000 1.2935 0.1709

Framework versions

  • Transformers 4.35.0
  • Pytorch 2.0.0
  • Datasets 2.14.6
  • Tokenizers 0.14.1
Downloads last month
1
Safetensors
Model size
94.4M params
Tensor type
F32
·

Finetuned from