Edit model card

Whisper Small Chinese MOE Response

This model is a fine-tuned version of Kathernie/whisper-small-all on the MOE Response Chinese dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2640
  • Cer: 11.0180

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 250
  • training_steps: 2000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Cer
0.2252 1.0811 200 0.2339 12.4230
0.1268 2.1622 400 0.2223 10.9194
0.056 3.2432 600 0.2242 10.8701
0.023 4.3243 800 0.2387 11.3384
0.01 5.4054 1000 0.2546 11.2645
0.0044 6.4865 1200 0.2515 11.2891
0.0028 7.5676 1400 0.2552 10.9440
0.0017 8.6486 1600 0.2623 11.2645
0.0017 9.7297 1800 0.2624 10.9933
0.001 10.8108 2000 0.2640 11.0180

Framework versions

  • Transformers 4.42.3
  • Pytorch 2.2.0
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
2
Safetensors
Model size
242M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for sit-justin/whisper-small-test

Finetuned
(1)
this model
Finetunes
1 model