Edit model card

wav2vec2-large-robust-ft-timit

This model is a fine-tuned version of facebook/wav2vec2-large-robust on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2768
  • Wer: 0.2321

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
6.6175 1.0 500 3.3025 1.0
3.0746 2.01 1000 2.9598 1.0
1.967 3.01 1500 0.6760 0.5607
0.7545 4.02 2000 0.4500 0.4567
0.5415 5.02 2500 0.3702 0.3882
0.4445 6.02 3000 0.3421 0.3584
0.3601 7.03 3500 0.2947 0.3096
0.3098 8.03 4000 0.2740 0.2894
0.2606 9.04 4500 0.2725 0.2787
0.238 10.04 5000 0.2549 0.2617
0.2142 11.04 5500 0.2485 0.2530
0.1787 12.05 6000 0.2683 0.2514
0.1652 13.05 6500 0.2559 0.2476
0.1569 14.06 7000 0.2777 0.2470
0.1443 15.06 7500 0.2661 0.2431
0.1335 16.06 8000 0.2717 0.2422
0.1291 17.07 8500 0.2672 0.2428
0.1192 18.07 9000 0.2684 0.2395
0.1144 19.08 9500 0.2770 0.2411
0.1052 20.08 10000 0.2831 0.2379
0.1004 21.08 10500 0.2847 0.2375
0.1053 22.09 11000 0.2851 0.2360
0.1005 23.09 11500 0.2807 0.2361
0.0904 24.1 12000 0.2764 0.2346
0.0876 25.1 12500 0.2774 0.2325
0.0883 26.1 13000 0.2768 0.2313
0.0848 27.11 13500 0.2840 0.2307
0.0822 28.11 14000 0.2812 0.2316
0.09 29.12 14500 0.2768 0.2321

Framework versions

  • Transformers 4.19.2
  • Pytorch 1.8.2+cu111
  • Datasets 1.17.0
  • Tokenizers 0.11.6
Downloads last month
2