MelanieKoe's picture
Model save
9257fce verified
metadata
license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v2-base-pretrained_lr5e-5_at0.2_da1
    results: []

w2v2-base-pretrained_lr5e-5_at0.2_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.3038
  • Wer: 0.1709

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • training_steps: 4000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
19.5617 3.91 250 4.1984 1.0
3.3686 7.81 500 3.2319 1.0
3.1228 11.72 750 3.1341 1.0
2.9603 15.62 1000 2.3654 1.0
1.0738 19.53 1250 0.7578 0.5485
0.3549 23.44 1500 0.6579 0.2337
0.2219 27.34 1750 0.8304 0.1999
0.1579 31.25 2000 0.9428 0.1828
0.1216 35.16 2250 1.0046 0.1747
0.0958 39.06 2500 1.0114 0.1751
0.076 42.97 2750 1.2645 0.1768
0.0662 46.88 3000 1.2588 0.1739
0.0569 50.78 3250 1.3057 0.1730
0.0514 54.69 3500 1.2869 0.1696
0.0479 58.59 3750 1.2697 0.1704
0.0451 62.5 4000 1.3038 0.1709

Framework versions

  • Transformers 4.35.0
  • Pytorch 2.0.0
  • Datasets 2.14.6
  • Tokenizers 0.14.1