Whisper Small Egyptian Arabic - Mohamed Abu El-Nasr

This model is a fine-tuned version of arbml/whisper-small-ar on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5173
  • Wer: 37.0920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-06
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 150
  • training_steps: 1920
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
2.0487 0.2510 60 1.6404 72.8783
1.1414 0.5021 120 1.0603 73.3333
0.8749 0.7531 180 0.8653 64.1543
0.7374 1.0042 240 0.6885 53.0366
0.5152 1.2552 300 0.5685 49.6142
0.4816 1.5063 360 0.5440 58.4965
0.4465 1.7573 420 0.5249 62.1563
0.4399 2.0084 480 0.5115 67.2404
0.3324 2.2594 540 0.5136 54.7774
0.3366 2.5105 600 0.5054 49.0010
0.3232 2.7615 660 0.4949 42.5717
0.3374 3.0126 720 0.4878 43.4817
0.2295 3.2636 780 0.4930 46.7062
0.2479 3.5146 840 0.4895 41.6617
0.2419 3.7657 900 0.4896 46.3699
0.2373 4.0167 960 0.4873 39.5846
0.1887 4.2678 1020 0.4961 37.0920
0.2069 4.5188 1080 0.4944 43.1652
0.1904 4.7699 1140 0.4954 39.9209
0.1856 5.0209 1200 0.4942 39.4857
0.1404 5.2720 1260 0.5025 45.1434
0.1706 5.5230 1320 0.5023 40.1978
0.1485 5.7741 1380 0.5034 40.6924
0.1568 6.0251 1440 0.5051 45.3412
0.1112 6.2762 1500 0.5109 44.9456
0.1311 6.5272 1560 0.5112 43.6400
0.1097 6.7782 1620 0.5129 46.5282
0.1333 7.0293 1680 0.5129 42.9080
0.1153 7.2803 1740 0.5167 43.1058
0.1059 7.5314 1800 0.5171 43.9763
0.116 7.7824 1860 0.5168 43.8773
0.1133 8.0335 1920 0.5173 42.4926

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
11
Safetensors
Model size
242M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for abuelnasr/whisper-small-eg

Finetuned
(3)
this model