xho_finetune / README.md
ivillar's picture
End of training
4855784 verified
metadata
license: apache-2.0
base_model: Akashpb13/Swahili_xlsr
tags:
  - generated_from_trainer
datasets:
  - ml-superb-subset
metrics:
  - wer
model-index:
  - name: xho_finetune
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: ml-superb-subset
          type: ml-superb-subset
          config: xho
          split: test
          args: xho
        metrics:
          - name: Wer
            type: wer
            value: 53.510895883777245

xho_finetune

This model is a fine-tuned version of Akashpb13/Swahili_xlsr on the ml-superb-subset dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5370
  • Wer: 53.5109

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 9.6e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 25
  • training_steps: 500
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
25.5184 0.7692 10 24.2275 100.0
14.5363 1.5385 20 9.8357 100.0
4.5811 2.3077 30 3.8367 100.0
3.4822 3.0769 40 3.3922 100.0
3.2732 3.8462 50 3.2398 100.0
3.1796 4.6154 60 3.1705 100.0
3.1504 5.3846 70 3.1419 100.0
3.1119 6.1538 80 3.1084 100.0
3.0789 6.9231 90 3.0735 100.0
3.0619 7.6923 100 3.0590 100.0
3.0298 8.4615 110 3.0247 100.0
2.9933 9.2308 120 2.9716 100.0
2.9079 10.0 130 2.8647 100.0
2.8414 10.7692 140 2.7931 100.0
2.6939 11.5385 150 2.5932 100.0
2.3274 12.3077 160 2.1000 99.7579
1.7068 13.0769 170 1.4580 93.4625
1.206 13.8462 180 1.1027 83.0508
0.9587 14.6154 190 0.9152 79.4189
0.7806 15.3846 200 0.8122 69.7337
0.7118 16.1538 210 0.7445 69.0073
0.6814 16.9231 220 0.6945 62.9540
0.5709 17.6923 230 0.6787 67.5545
0.5653 18.4615 240 0.6758 62.2276
0.5437 19.2308 250 0.6511 60.7748
0.5092 20.0 260 0.6237 62.7119
0.4239 20.7692 270 0.6000 61.5012
0.4355 21.5385 280 0.5899 59.8063
0.4456 22.3077 290 0.5960 59.3220
0.3986 23.0769 300 0.5764 56.6586
0.3856 23.8462 310 0.5801 55.9322
0.3607 24.6154 320 0.5682 57.6271
0.358 25.3846 330 0.5675 55.9322
0.3452 26.1538 340 0.5630 57.8692
0.3289 26.9231 350 0.5515 57.8692
0.353 27.6923 360 0.5621 57.3850
0.2907 28.4615 370 0.5486 55.2058
0.3237 29.2308 380 0.5445 54.4794
0.3202 30.0 390 0.5384 52.7845
0.2918 30.7692 400 0.5370 55.6901
0.3106 31.5385 410 0.5422 53.7530
0.3105 32.3077 420 0.5438 55.2058
0.2835 33.0769 430 0.5437 55.9322
0.2966 33.8462 440 0.5416 54.7215
0.2719 34.6154 450 0.5394 54.2373
0.2859 35.3846 460 0.5384 53.7530
0.29 36.1538 470 0.5379 53.2688
0.2879 36.9231 480 0.5372 53.5109
0.2871 37.6923 490 0.5370 53.5109
0.3019 38.4615 500 0.5370 53.5109

Framework versions

  • Transformers 4.41.1
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1