whisper-small-khmer-v7

This model is a fine-tuned version of openai/whisper-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3125
• Wer: 96.7938

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.6143	1.0	984	0.2394	103.3167
0.2526	2.0	1968	0.1995	127.2803
0.1961	3.0	2952	0.1961	97.0426
0.1651	4.0	3936	0.1881	96.9873
0.1443	5.0	4920	0.1940	95.4671
0.1278	6.0	5904	0.1976	95.5777
0.1125	7.0	6888	0.2069	97.4848
0.1003	8.0	7872	0.2080	94.1680
0.0905	9.0	8856	0.2160	94.4168
0.0808	10.0	9840	0.2313	94.9143
0.0743	11.0	10824	0.2313	94.2786
0.0673	12.0	11808	0.2426	96.1028
0.0626	13.0	12792	0.2378	96.0475
0.0579	14.0	13776	0.2428	95.4671
0.0538	15.0	14760	0.2484	95.6329
0.051	16.0	15744	0.2619	95.1907
0.0482	17.0	16728	0.2618	94.7761
0.0455	18.0	17712	0.2732	95.3842
0.0434	19.0	18696	0.2746	94.4997
0.0416	20.0	19680	0.2761	94.5826
0.0394	21.0	20664	0.2770	94.2510
0.0385	22.0	21648	0.2841	94.4721
0.0369	23.0	22632	0.2813	95.5224
0.0355	24.0	23616	0.2876	93.8364
0.0338	25.0	24600	0.2894	96.0475
0.0334	26.0	25584	0.2915	94.4168
0.0319	27.0	26568	0.2886	95.6329
0.0307	28.0	27552	0.3046	98.2311
0.0299	29.0	28536	0.3046	95.1354
0.0291	30.0	29520	0.3125	96.7938

Framework versions

• Transformers 4.47.0
• Pytorch 2.3.1
• Datasets 3.1.0
• Tokenizers 0.21.0