whisper-small-moore-v2

This model is a fine-tuned version of openai/whisper-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5124
• Wer: 40.4014

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.0637	0.9470	500	0.5086	48.3004
0.6753	1.8939	1000	0.4204	41.2431
0.3738	2.8409	1500	0.4010	40.6280
0.2418	3.7879	2000	0.4068	39.0741
0.1319	4.7348	2500	0.4307	39.2036
0.0722	5.6818	3000	0.4523	39.0741
0.0410	6.6288	3500	0.4703	39.9482
0.0203	7.5758	4000	0.4912	40.2396
0.0088	8.5227	4500	0.5065	39.7216
0.0062	9.4697	5000	0.5124	40.4014

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2