ymoslem's picture
End of training
9113038 verified
metadata
language:
  - ga
  - en
license: apache-2.0
base_model: openai/whisper-medium
tags:
  - generated_from_trainer
datasets:
  - ymoslem/IWSLT2023-GA-EN
  - ymoslem/FLEURS-GA-EN
  - ymoslem/BitesizeIrish-GA-EN
  - ymoslem/SpokenWords-GA-EN-MTed
  - ymoslem/Tatoeba-Speech-Irish
  - ymoslem/Wikimedia-Speech-Irish
metrics:
  - bleu
  - wer
model-index:
  - name: Whisper Small GA-EN Speech Translation
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: IWSLT-2023, FLEURS, BiteSize, SpokenWords, Tatoeba, and Wikimedia
          type: ymoslem/IWSLT2023-GA-EN
        metrics:
          - name: Bleu
            type: bleu
            value: 33.77
          - name: Wer
            type: wer
            value: 60.828455650607836

Whisper Small GA-EN Speech Translation

This model is a fine-tuned version of openai/whisper-medium on the IWSLT-2023, FLEURS, BiteSize, SpokenWords, Tatoeba, and Wikimedia dataset. It achieves the following results on the evaluation set:

  • Loss: 1.2028
  • Bleu: 33.77
  • Chrf: 52.79
  • Wer: 60.8285

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.03
  • training_steps: 4000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Bleu Chrf Wer
2.4145 0.0109 100 2.1019 2.32 16.08 170.5088
2.6073 0.0219 200 2.0370 5.94 23.77 158.8924
2.593 0.0328 300 1.8529 3.67 21.53 238.6312
2.3123 0.0438 400 1.8500 9.01 28.13 135.0293
2.3347 0.0547 500 1.7816 15.05 31.9 90.7249
2.1277 0.0657 600 1.6916 14.24 32.29 88.4286
2.1836 0.0766 700 1.6517 12.15 32.7 128.1405
2.0112 0.0876 800 1.6275 19.76 38.15 79.2886
1.8387 0.0985 900 1.6349 17.26 38.82 91.0851
1.8335 0.1095 1000 1.5843 20.93 38.02 75.9118
1.7849 0.1204 1100 1.5863 15.98 37.5 96.5781
1.5698 0.1314 1200 1.5371 16.42 39.07 103.6020
1.4759 0.1423 1300 1.5250 18.56 38.41 96.5781
1.4915 0.1533 1400 1.4862 22.05 40.15 75.1013
1.6583 0.1642 1500 1.4727 18.11 39.65 95.7677
1.3981 0.1752 1600 1.4367 27.31 44.5 66.0513
1.2646 0.1861 1700 1.4574 22.85 42.19 74.4710
1.2172 0.1970 1800 1.3818 20.77 42.5 82.7105
1.183 0.2080 1900 1.4380 22.75 41.28 76.7672
1.1931 0.2189 2000 1.3917 23.58 41.13 77.3075
1.172 0.2299 2100 1.3892 24.58 44.4 74.3809
1.0284 0.2408 2200 1.3806 23.34 44.1 78.0279
0.8507 0.2518 2300 1.3210 28.67 46.79 67.1769
0.9615 0.2627 2400 1.3103 27.95 46.8 70.0135
0.8049 0.2737 2500 1.3141 29.92 48.9 67.2220
0.7639 0.2846 2600 1.3085 30.91 49.05 64.2053
0.8594 0.2956 2700 1.3378 27.8 47.84 68.8879
0.7482 0.3065 2800 1.2978 30.6 48.62 64.9257
0.6941 0.3175 2900 1.3060 29.92 47.92 65.8712
0.7282 0.3284 3000 1.2959 31.09 48.13 65.3309
0.6298 0.3394 3100 1.2893 29.76 48.8 67.1769
0.619 0.3503 3200 1.2388 32.61 50.27 62.0891
0.6252 0.3612 3300 1.2550 32.71 50.96 62.4493
0.4699 0.3722 3400 1.2463 32.02 51.24 65.2409
0.5121 0.3831 3500 1.2214 32.26 51.29 63.7551
0.5092 0.3941 3600 1.2182 32.88 51.59 62.0891
0.4365 0.4050 3700 1.2049 32.16 51.5 62.3143
0.2971 0.4160 3800 1.2201 34.45 52.78 59.7479
0.389 0.4269 3900 1.2007 33.86 53.28 60.6033
0.3879 0.4379 4000 1.2028 33.77 52.79 60.8285

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.2.0+cu121
  • Datasets 2.19.2
  • Tokenizers 0.19.1