Edit model card

hubert-base-timit-demo-google-colab-ft35ep

This model is a fine-tuned version of facebook/hubert-base-ls960 on the timit-asr dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4602
  • Wer: 0.3466

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
3.825 0.87 500 2.9521 1.0
2.431 1.73 1000 0.9760 0.8013
1.0089 2.6 1500 0.5934 0.5968
0.6859 3.46 2000 0.5132 0.5356
0.5302 4.33 2500 0.4506 0.4894
0.44 5.19 3000 0.4340 0.4670
0.3926 6.06 3500 0.4506 0.4528
0.3326 6.92 4000 0.4197 0.4486
0.2937 7.79 4500 0.4093 0.4193
0.2568 8.65 5000 0.4098 0.4229
0.2473 9.52 5500 0.4090 0.4141
0.2233 10.38 6000 0.4152 0.4125
0.2108 11.25 6500 0.4586 0.4189
0.2086 12.11 7000 0.4284 0.3969
0.1858 12.98 7500 0.4028 0.3946
0.1641 13.84 8000 0.4679 0.4002
0.1686 14.71 8500 0.4441 0.3936
0.1489 15.57 9000 0.4897 0.3828
0.1541 16.44 9500 0.4953 0.3783
0.1417 17.3 10000 0.4500 0.3758
0.1428 18.17 10500 0.4533 0.3796
0.1306 19.03 11000 0.4474 0.3792
0.1185 19.9 11500 0.4762 0.3743
0.1081 20.76 12000 0.4770 0.3699
0.1253 21.63 12500 0.4749 0.3629
0.1087 22.49 13000 0.4577 0.3534
0.1172 23.36 13500 0.4819 0.3525
0.1086 24.22 14000 0.4709 0.3623
0.089 25.09 14500 0.4852 0.3544
0.086 25.95 15000 0.4602 0.3555
0.086 26.82 15500 0.4861 0.3497
0.086 27.68 16000 0.4527 0.3473
0.0919 28.55 16500 0.4607 0.3487
0.0792 29.41 17000 0.4602 0.3466

Framework versions

  • Transformers 4.17.0
  • Pytorch 1.11.0+cu113
  • Datasets 1.18.3
  • Tokenizers 0.12.1
Downloads last month
7
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.