MelanieKoe's picture
Model save
8b17fc6 verified
metadata
license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v2-base-pretrained_lr5e-5_at0.8_da1
    results: []

w2v2-base-pretrained_lr5e-5_at0.8_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.6691
  • Wer: 0.1858

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • training_steps: 4000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
14.0341 5.43 250 3.5667 1.0
3.138 10.87 500 3.1976 1.0
3.0471 16.3 750 3.0452 1.0
1.3116 21.74 1000 1.0225 0.3358
0.1806 27.17 1250 1.2151 0.2533
0.1065 32.61 1500 1.2673 0.2384
0.0707 38.04 1750 1.4184 0.1888
0.0547 43.48 2000 1.6087 0.1901
0.0471 48.91 2250 1.4537 0.1880
0.0381 54.35 2500 1.6858 0.1845
0.0307 59.78 2750 1.5607 0.1961
0.0269 65.22 3000 1.6716 0.1862
0.024 70.65 3250 1.5554 0.1884
0.0199 76.09 3500 1.7169 0.1850
0.0187 81.52 3750 1.6661 0.1858
0.0176 86.96 4000 1.6691 0.1858

Framework versions

  • Transformers 4.35.0
  • Pytorch 2.0.0
  • Datasets 2.14.6
  • Tokenizers 0.14.1