Aichunks's picture
End of training
ffd0398 verified
metadata
license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: wav2vec2-base-timit-demo-google-colab
    results: []

wav2vec2-base-timit-demo-google-colab

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5313
  • Wer: 0.3317

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
3.5823 1.0 500 1.8501 1.0236
0.8931 2.01 1000 0.5018 0.5196
0.4269 3.01 1500 0.4266 0.4461
0.2876 4.02 2000 0.4458 0.4359
0.2272 5.02 2500 0.4183 0.4146
0.1813 6.02 3000 0.4151 0.3945
0.1555 7.03 3500 0.4216 0.3881
0.1353 8.03 4000 0.4282 0.3824
0.1221 9.04 4500 0.4848 0.3845
0.1135 10.04 5000 0.5003 0.3818
0.0968 11.04 5500 0.5331 0.3738
0.09 12.05 6000 0.5082 0.3690
0.084 13.05 6500 0.4573 0.3634
0.0744 14.06 7000 0.4711 0.3705
0.0663 15.06 7500 0.4955 0.3634
0.0612 16.06 8000 0.4721 0.3558
0.0535 17.07 8500 0.4965 0.3654
0.0527 18.07 9000 0.5381 0.3592
0.0458 19.08 9500 0.5029 0.3498
0.0424 20.08 10000 0.5814 0.3547
0.042 21.08 10500 0.4893 0.3480
0.0373 22.09 11000 0.5047 0.3482
0.0333 23.09 11500 0.5235 0.3426
0.0306 24.1 12000 0.5165 0.3472
0.0293 25.1 12500 0.4988 0.3426
0.025 26.1 13000 0.5157 0.3382
0.0255 27.11 13500 0.5278 0.3412
0.022 28.11 14000 0.5401 0.3364
0.0195 29.12 14500 0.5313 0.3317

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 1.18.3
  • Tokenizers 0.15.1