arun100/whisper-base-bn-7

Whisper Base Bengali

This model is a fine-tuned version of arun100/whisper-base-bn-6 on the mozilla-foundation/common_voice_16_0 bn dataset. It achieves the following results on the evaluation set:

• Loss: 0.2028
• Wer: 27.7210

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1388	1.72	500	0.2076	28.8529
0.1335	3.43	1000	0.2068	28.6171
0.1297	5.15	1500	0.2061	28.4462
0.13	6.86	2000	0.2050	28.4274
0.126	8.58	2500	0.2046	28.3500
0.122	10.29	3000	0.2044	28.1872
0.1205	12.01	3500	0.2039	28.1715
0.1187	13.72	4000	0.2038	28.0136
0.1152	15.44	4500	0.2035	28.0503
0.1133	17.15	5000	0.2035	28.0395
0.1167	18.87	5500	0.2031	27.9071
0.1119	20.58	6000	0.2032	27.8288
0.1168	22.3	6500	0.2030	27.8306
0.1129	24.01	7000	0.2029	27.7778
0.112	25.73	7500	0.2030	27.7415
0.1105	27.44	8000	0.2030	27.7482
0.1114	29.16	8500	0.2028	27.7608
0.1102	30.87	9000	0.2029	27.7357
0.1115	32.59	9500	0.2028	27.7210
0.1083	34.31	10000	0.2028	27.7353

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0