Whisper small FTSpeech - Julie

This model is a fine-tuned version of openai/whisper-large on the ftspeech dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2781
  • Wer: 19.4638

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 200
  • training_steps: 5000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
0.4214 0.0080 500 0.4317 26.8590
0.3568 0.0161 1000 0.3763 24.5151
0.3443 0.0241 1500 0.3443 23.0618
0.3218 0.0321 2000 0.3275 22.0048
0.2851 0.0402 2500 0.3139 21.2409
0.2638 0.0482 3000 0.3021 20.4187
0.2515 0.0562 3500 0.2943 20.2420
0.2692 0.0643 4000 0.2864 19.9020
0.2503 0.0723 4500 0.2806 19.6671
0.2396 0.0803 5000 0.2781 19.4638

Framework versions

  • Transformers 4.47.0
  • Pytorch 2.5.1
  • Datasets 3.1.0
  • Tokenizers 0.21.0
Downloads last month
130
Safetensors
Model size
242M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for JulieHinge/whisper-small-ftspeech

Finetuned
(55)
this model

Dataset used to train JulieHinge/whisper-small-ftspeech

Evaluation results