Edit model card

libri-smallw2v2-no-copy-mse-alpha-0.75-T-1-take-5

This model is a fine-tuned version of rohitp1/libri-smallw2v2-no-copy-mse-alpha-0.75-T-1-take-3 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 30.3050
  • Wer: 0.2650

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.002
  • train_batch_size: 4
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.2
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
239.4865 1.12 400 31.0836 0.2908
210.2046 2.25 800 29.7831 0.2742
195.0478 3.37 1200 28.8794 0.2636
188.8096 4.49 1600 28.7458 0.2600
183.6592 5.62 2000 29.1159 0.2573
181.5025 6.74 2400 29.0081 0.2564
181.9954 7.86 2800 28.7132 0.2588
181.8548 8.99 3200 29.3207 0.2630
186.2524 10.11 3600 29.9119 0.2593
188.3838 11.24 4000 30.2963 0.2627
192.2623 12.36 4400 30.3050 0.2650

Framework versions

  • Transformers 4.24.0
  • Pytorch 1.12.1
  • Datasets 2.7.1
  • Tokenizers 0.11.0
Downloads last month
8