Gabi00/distil_whisper-v3-LoRA-en_students_test

Whisper Small Eng - Gabriel Mora

This model is a fine-tuned version of openai/whisper-small on the English-mistakes dataset. It achieves the following results on the evaluation set:

• Loss: 0.6550
• Wer: 18.2337

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 28
• eval_batch_size: 28
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• training_steps: 100000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.5085	0.4444	500	1.1844	25.9507
1.1717	0.8889	1000	0.9522	25.2751
1.1302	1.3333	1500	0.8634	22.0879
1.0094	1.7778	2000	0.8098	21.0103
1.0509	2.2222	2500	0.7784	23.2054
0.9722	2.6667	3000	0.7555	21.5206
0.9562	3.1111	3500	0.7401	21.0075
0.9995	3.5556	4000	0.7269	19.8985
0.9497	4.0	4500	0.7170	19.3626
0.8703	4.4444	5000	0.7078	19.4652
1.0015	4.8889	5500	0.7004	20.1608
0.9248	5.3333	6000	0.6947	17.7034
0.9163	5.7778	6500	0.6880	17.4953
0.8833	6.2222	7000	0.6823	17.4668
0.9051	6.6667	7500	0.6770	17.4554
0.8882	7.1111	8000	0.6730	17.3613
0.8879	7.5556	8500	0.6684	18.3220
0.8396	8.0	9000	0.6647	18.2165
0.9282	8.4444	9500	0.6616	18.4646
0.8581	8.8889	10000	0.6578	18.1538
0.8938	9.3333	10500	0.6550	18.2337

Framework versions

• PEFT 0.11.1
• Transformers 4.42.3
• Pytorch 2.1.0+cu118
• Datasets 2.20.0
• Tokenizers 0.19.1