MelanieKoe's picture
Model save
fb39889 verified
metadata
license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v2-base-pretrained_lr5e-5_at0.6_da1
    results: []

w2v2-base-pretrained_lr5e-5_at0.6_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4078
  • Wer: 0.1662

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • training_steps: 4000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
25.5608 4.24 250 20.1943 1.0
5.6218 8.47 500 3.4933 1.0
3.134 12.71 750 3.1332 1.0
3.0842 16.95 1000 3.1196 1.0
2.6714 21.19 1250 1.4095 0.9932
0.5642 25.42 1500 0.6028 0.4327
0.2724 29.66 1750 0.7355 0.2396
0.1676 33.9 2000 0.9693 0.2264
0.1247 38.14 2250 1.0615 0.2008
0.0945 42.37 2500 1.2612 0.1841
0.0784 46.61 2750 1.1492 0.1850
0.0672 50.85 3000 1.3113 0.1751
0.0567 55.08 3250 1.3594 0.1721
0.0507 59.32 3500 1.4195 0.1721
0.0478 63.56 3750 1.3916 0.1670
0.0445 67.8 4000 1.4078 0.1662

Framework versions

  • Transformers 4.35.0
  • Pytorch 2.0.0
  • Datasets 2.14.6
  • Tokenizers 0.14.1