razhan/whisper-small-ckb

Whisper Small Ckb - Razhan Hameed

This model is a fine-tuned version of openai/whisper-small on the Common Voice 11.0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.3825
• Wer: 33.2193

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 64
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 12000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1693	2.49	1000	0.2060	39.1265
0.0722	4.98	2000	0.2124	36.3173
0.0127	7.46	3000	0.2736	36.5568
0.008	9.95	4000	0.3131	35.7015
0.0032	12.44	5000	0.3434	35.3936
0.0028	14.93	6000	0.3453	35.9258
0.003	17.41	7000	0.3558	34.9565
0.0022	19.9	8000	0.3593	34.2722
0.0016	22.39	9000	0.3639	34.3369
0.0015	24.88	10000	0.3785	34.0062
0.0009	27.36	11000	0.3915	34.2951
0.0001	29.85	12000	0.3825	33.2193

Framework versions

• Transformers 4.26.0.dev0
• Pytorch 1.13.0+cu117
• Datasets 2.7.1.dev0
• Tokenizers 0.13.2