Edit model card

hubert-base-libri-demo-feature_extractor_not_frozen_v4_25epochs_weight_decay

This model is a fine-tuned version of facebook/hubert-base-ls960 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.1189
  • Wer: 0.1105

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.00015
  • train_batch_size: 64
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 3000
  • num_epochs: 25
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
4.3661 1.12 500 3.4415 0.9837
2.8763 2.24 1000 3.1292 0.9837
1.6666 3.36 1500 0.5027 0.4804
0.4759 4.48 2000 0.2187 0.2536
0.2774 5.61 2500 0.1555 0.1898
0.2026 6.73 3000 0.1297 0.1543
0.1745 7.85 3500 0.1201 0.1419
0.1596 8.97 4000 0.1220 0.1339
0.1449 10.09 4500 0.1156 0.1280
0.1134 11.21 5000 0.1131 0.1244
0.1143 12.33 5500 0.1189 0.1226
0.0915 13.45 6000 0.1138 0.1196
0.0904 14.57 6500 0.1125 0.1195
0.0853 15.7 7000 0.1125 0.1168
0.0775 16.82 7500 0.1103 0.1155
0.0732 17.94 8000 0.1115 0.1138
0.0728 19.06 8500 0.1196 0.1142
0.0755 20.18 9000 0.1170 0.1122
0.0647 21.3 9500 0.1167 0.1117
0.064 22.42 10000 0.1177 0.1109
0.0591 23.54 10500 0.1182 0.1110
0.0566 24.66 11000 0.1189 0.1105

Framework versions

  • Transformers 4.30.0.dev0
  • Pytorch 2.0.1
  • Datasets 2.12.1.dev0
  • Tokenizers 0.13.3
Downloads last month
1
Inference API
or
This model can be loaded on Inference API (serverless).