whisper-small-pashto

This model is a fine-tuned version of ihanif/whisper_small_ps_augmented on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2602
• Wer: 27.2082

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 20
• training_steps: 400
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.0635	10	0.2723	28.0859
No log	0.1270	20	0.2716	27.6377
0.2713	0.1905	30	0.2703	27.7311
0.2713	0.2540	40	0.2692	27.3950
0.2851	0.3175	50	0.2681	27.2082
0.2851	0.3810	60	0.2667	27.0588
0.2851	0.4444	70	0.2657	26.9094
0.303	0.5079	80	0.2651	26.8908
0.303	0.5714	90	0.2645	26.8161
0.2649	0.6349	100	0.2641	26.9841
0.2649	0.6984	110	0.2635	26.9841
0.2649	0.7619	120	0.2627	27.1895
0.2848	0.8254	130	0.2622	27.1895
0.2848	0.8889	140	0.2617	27.1522
0.2855	0.9524	150	0.2609	27.2082
0.2855	1.0159	160	0.2604	26.9655
0.2855	1.0794	170	0.2602	27.2082

Framework versions

• PEFT 0.12.0
• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 4.8.5
• Tokenizers 0.19.1