arun100/whisper-base-uk-1

Whisper Base Ukrainian

This model is a fine-tuned version of openai/whisper-base on the mozilla-foundation/common_voice_16_0 uk dataset. It achieves the following results on the evaluation set:

• Loss: 0.4984
• Wer: 35.9409

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.5338	3.01	1000	0.5985	42.0862
0.4086	6.02	2000	0.5545	39.8889
0.336	9.02	3000	0.5347	38.2399
0.3672	13.0	4000	0.5165	37.2395
0.3862	16.01	5000	0.5096	36.7576
0.3298	19.02	6000	0.5040	36.5556
0.297	22.03	7000	0.5040	36.3153
0.3006	26.0	8000	0.4985	36.0172
0.3429	29.01	9000	0.4984	35.9409
0.2978	32.02	10000	0.4978	35.9423

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0