whisper-large-cit-do0.15-wd0.0001

This model is a fine-tuned version of openai/whisper-large-v3 on the SF 200 dataset. It achieves the following results on the evaluation set:

• Loss: 0.6948
• Wer: 34.0961

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 200
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.1267	0.8889	10	1.1143	48.9703
1.0883	1.7778	20	1.0068	40.0458
0.9315	2.6667	30	0.8667	38.9016
0.751	3.5556	40	0.7886	34.0961
0.6968	4.4444	50	0.7158	35.0114
0.5946	5.3333	60	0.6504	31.8078
0.5011	6.2222	70	0.6133	31.3501
0.4061	7.1111	80	0.5869	33.6384
0.3731	8.0	90	0.5718	32.9519
0.2977	8.8889	100	0.5688	33.1808
0.2612	9.7778	110	0.5742	32.9519
0.2105	10.6667	120	0.5845	32.2654
0.1775	11.5556	130	0.5981	32.7231
0.1479	12.4444	140	0.6118	31.1213
0.1173	13.3333	150	0.6255	33.1808
0.111	14.2222	160	0.6426	35.4691
0.0946	15.1111	170	0.6641	34.5538
0.0799	16.0	180	0.6772	34.7826
0.0739	16.8889	190	0.6904	34.0961
0.0682	17.7778	200	0.6948	34.0961

Framework versions

• Transformers 4.41.1
• Pytorch 1.13.1+cu117
• Datasets 2.19.1
• Tokenizers 0.19.1