markredito/whisper-small-dv

Whisper Small Dv - Mark Redito

This model is a fine-tuned version of openai/whisper-small on the Common Voice 13 - Dhivehi dataset. It achieves the following results on the evaluation set:

• Loss: 0.3041
• Wer Ortho: 56.8563
• Wer: 10.8634

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
0.121	1.6287	500	0.1725	62.6994	13.3481
0.0435	3.2573	1000	0.1651	57.9567	11.5659
0.0296	4.8860	1500	0.1829	58.1517	11.6076
0.0071	6.5147	2000	0.2369	58.8203	11.4198
0.0016	8.1433	2500	0.2718	57.9915	11.2129
0.0011	9.7720	3000	0.2923	57.1349	11.1242
0.0002	11.4007	3500	0.3001	56.8633	10.9173
0.0002	13.0293	4000	0.3041	56.8563	10.8634

Framework versions

• Transformers 4.42.4
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1