simpragma/breeze-dsw-tiny-ml

Breeze DSW Malayalam - tiny

This model is a fine-tuned version of openai/whisper-tiny on the mozilla-foundation/common_voice_16_0 ml dataset. It achieves the following results on the evaluation set:

• Loss: 0.5503
• Wer: 54.3744

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 2000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.1736	2.02	100	1.1670	99.7776
0.9647	4.04	200	1.0049	95.4866
0.5311	7.02	300	0.6807	74.5598
0.3036	9.04	400	0.5410	61.5755
0.1672	12.02	500	0.5146	56.5709
0.1006	14.04	600	0.5503	54.3744
0.0484	17.02	700	0.5859	54.5042
0.0305	19.04	800	0.6562	55.4124
0.0147	22.02	900	0.7095	54.8749
0.0116	24.04	1000	0.7383	55.0973

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0