abuelnasr/whisper-small-eg

Whisper Small Egyptian Arabic - Mohamed Abu El-Nasr

This model is a fine-tuned version of arbml/whisper-small-ar on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5173
• Wer: 37.0920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 150
• training_steps: 1920
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.0487	0.2510	60	1.6404	72.8783
1.1414	0.5021	120	1.0603	73.3333
0.8749	0.7531	180	0.8653	64.1543
0.7374	1.0042	240	0.6885	53.0366
0.5152	1.2552	300	0.5685	49.6142
0.4816	1.5063	360	0.5440	58.4965
0.4465	1.7573	420	0.5249	62.1563
0.4399	2.0084	480	0.5115	67.2404
0.3324	2.2594	540	0.5136	54.7774
0.3366	2.5105	600	0.5054	49.0010
0.3232	2.7615	660	0.4949	42.5717
0.3374	3.0126	720	0.4878	43.4817
0.2295	3.2636	780	0.4930	46.7062
0.2479	3.5146	840	0.4895	41.6617
0.2419	3.7657	900	0.4896	46.3699
0.2373	4.0167	960	0.4873	39.5846
0.1887	4.2678	1020	0.4961	37.0920
0.2069	4.5188	1080	0.4944	43.1652
0.1904	4.7699	1140	0.4954	39.9209
0.1856	5.0209	1200	0.4942	39.4857
0.1404	5.2720	1260	0.5025	45.1434
0.1706	5.5230	1320	0.5023	40.1978
0.1485	5.7741	1380	0.5034	40.6924
0.1568	6.0251	1440	0.5051	45.3412
0.1112	6.2762	1500	0.5109	44.9456
0.1311	6.5272	1560	0.5112	43.6400
0.1097	6.7782	1620	0.5129	46.5282
0.1333	7.0293	1680	0.5129	42.9080
0.1153	7.2803	1740	0.5167	43.1058
0.1059	7.5314	1800	0.5171	43.9763
0.116	7.7824	1860	0.5168	43.8773
0.1133	8.0335	1920	0.5173	42.4926

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1