hubert_model
This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.2139
- Wer: 1.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 64
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 125
- num_epochs: 30
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
55.5107 | 0.11 | 100 | 93.6947 | 1.0 |
29.8329 | 0.22 | 200 | 53.0718 | 1.0 |
22.4958 | 0.32 | 300 | 42.6961 | 1.0 |
19.1734 | 0.43 | 400 | 34.1686 | 1.0 |
15.9615 | 0.54 | 500 | 27.1054 | 1.0 |
13.1077 | 0.65 | 600 | 21.2901 | 1.0 |
11.0162 | 0.76 | 700 | 16.6558 | 1.0 |
9.3359 | 0.87 | 800 | 13.1283 | 1.0 |
8.2754 | 0.97 | 900 | 10.6005 | 1.0 |
7.1321 | 1.08 | 1000 | 8.7120 | 1.0 |
6.2621 | 1.19 | 1100 | 7.4866 | 1.0 |
5.8109 | 1.3 | 1200 | 6.6416 | 1.0 |
5.386 | 1.41 | 1300 | 6.1307 | 1.0 |
5.1782 | 1.51 | 1400 | 5.8103 | 1.0 |
4.9481 | 1.62 | 1500 | 5.6119 | 1.0 |
4.8722 | 1.73 | 1600 | 5.4872 | 1.0 |
4.7617 | 1.84 | 1700 | 5.3270 | 1.0 |
4.717 | 1.95 | 1800 | 5.2877 | 1.0 |
4.6256 | 2.06 | 1900 | 5.6727 | 1.0 |
4.6255 | 2.16 | 2000 | 5.4983 | 1.0 |
4.5977 | 2.27 | 2100 | 5.2167 | 1.0 |
4.5797 | 2.38 | 2200 | 4.9743 | 1.0 |
4.5616 | 2.49 | 2300 | 4.8446 | 1.0 |
4.5476 | 2.6 | 2400 | 4.7885 | 1.0 |
4.5516 | 2.71 | 2500 | 4.7597 | 1.0 |
4.5343 | 2.81 | 2600 | 4.7309 | 1.0 |
4.586 | 2.92 | 2700 | 4.7173 | 1.0 |
4.5813 | 3.03 | 2800 | 4.6650 | 1.0 |
4.4794 | 3.14 | 2900 | 4.5851 | 1.0 |
4.4735 | 3.25 | 3000 | 4.5310 | 1.0 |
4.4748 | 3.35 | 3100 | 4.5285 | 1.0 |
4.4439 | 3.46 | 3200 | 4.4971 | 1.0 |
4.4255 | 3.57 | 3300 | 4.5072 | 1.0 |
4.4232 | 3.68 | 3400 | 4.4936 | 1.0 |
4.4066 | 3.79 | 3500 | 4.5279 | 1.0 |
4.4095 | 3.9 | 3600 | 4.4653 | 1.0 |
4.3148 | 4.0 | 3700 | 4.4542 | 1.0 |
4.2788 | 4.11 | 3800 | 4.3820 | 1.0 |
4.3291 | 4.22 | 3900 | 4.3234 | 1.0 |
4.2173 | 4.33 | 4000 | 4.3068 | 1.0 |
4.1921 | 4.44 | 4100 | 4.2719 | 1.0 |
4.1868 | 4.55 | 4200 | 4.2765 | 1.0 |
4.1734 | 4.65 | 4300 | 4.2349 | 1.0 |
4.1868 | 4.76 | 4400 | 4.2002 | 1.0 |
4.169 | 4.87 | 4500 | 4.1915 | 1.0 |
4.131 | 4.98 | 4600 | 4.1673 | 1.0 |
4.1952 | 5.09 | 4700 | 4.1657 | 1.0 |
4.1067 | 5.19 | 4800 | 4.1650 | 1.0 |
4.1026 | 5.3 | 4900 | 4.1394 | 1.0 |
4.0864 | 5.41 | 5000 | 4.1334 | 1.0 |
4.0745 | 5.52 | 5100 | 4.1138 | 1.0 |
4.0653 | 5.63 | 5200 | 4.1029 | 1.0 |
4.0484 | 5.74 | 5300 | 4.0870 | 1.0 |
4.0474 | 5.84 | 5400 | 4.0693 | 1.0 |
4.0299 | 5.95 | 5500 | 4.0489 | 1.0 |
4.0862 | 6.06 | 5600 | 4.0414 | 1.0 |
3.9986 | 6.17 | 5700 | 4.0316 | 1.0 |
4.0042 | 6.28 | 5800 | 4.0100 | 1.0 |
3.9912 | 6.39 | 5900 | 3.9861 | 1.0 |
3.9625 | 6.49 | 6000 | 3.9676 | 1.0 |
3.96 | 6.6 | 6100 | 3.9469 | 1.0 |
3.9443 | 6.71 | 6200 | 3.9514 | 1.0 |
3.9215 | 6.82 | 6300 | 3.9108 | 1.0 |
3.9176 | 6.93 | 6400 | 3.8880 | 1.0 |
3.9986 | 7.03 | 6500 | 3.8798 | 1.0 |
3.8908 | 7.14 | 6600 | 3.8610 | 1.0 |
3.8715 | 7.25 | 6700 | 3.8430 | 1.0 |
3.8751 | 7.36 | 6800 | 3.8144 | 1.0 |
3.8643 | 7.47 | 6900 | 3.7939 | 1.0 |
3.8325 | 7.58 | 7000 | 3.7716 | 1.0 |
3.8269 | 7.68 | 7100 | 3.7620 | 1.0 |
3.82 | 7.79 | 7200 | 3.7440 | 1.0 |
3.8037 | 7.9 | 7300 | 3.7141 | 1.0 |
3.7488 | 8.01 | 7400 | 3.6912 | 1.0 |
3.7706 | 8.12 | 7500 | 3.6651 | 1.0 |
3.7454 | 8.22 | 7600 | 3.6520 | 1.0 |
3.748 | 8.33 | 7700 | 3.6190 | 1.0 |
3.7375 | 8.44 | 7800 | 3.6024 | 1.0 |
3.7045 | 8.55 | 7900 | 3.5830 | 1.0 |
3.6915 | 8.66 | 8000 | 3.5455 | 1.0 |
3.6647 | 8.77 | 8100 | 3.5333 | 1.0 |
3.645 | 8.87 | 8200 | 3.5053 | 1.0 |
3.6229 | 8.98 | 8300 | 3.4728 | 1.0 |
3.6574 | 9.09 | 8400 | 3.4310 | 1.0 |
3.6235 | 9.2 | 8500 | 3.4228 | 1.0 |
3.5773 | 9.31 | 8600 | 3.3695 | 1.0 |
3.5876 | 9.42 | 8700 | 3.3636 | 1.0 |
3.5547 | 9.52 | 8800 | 3.3299 | 1.0 |
3.5691 | 9.63 | 8900 | 3.3324 | 1.0 |
3.5284 | 9.74 | 9000 | 3.2827 | 1.0 |
3.4919 | 9.85 | 9100 | 3.2855 | 1.0 |
3.4769 | 9.96 | 9200 | 3.2446 | 1.0 |
3.4516 | 10.06 | 9300 | 3.2290 | 1.0 |
3.4402 | 10.17 | 9400 | 3.2170 | 1.0 |
3.3962 | 10.28 | 9500 | 3.1936 | 1.0 |
3.4377 | 10.39 | 9600 | 3.1687 | 1.0 |
3.3816 | 10.5 | 9700 | 3.1436 | 1.0 |
3.3902 | 10.61 | 9800 | 3.1505 | 1.0 |
3.4016 | 10.71 | 9900 | 3.1450 | 1.0 |
3.3716 | 10.82 | 10000 | 3.1074 | 1.0 |
3.3278 | 10.93 | 10100 | 3.0856 | 1.0 |
3.3598 | 11.04 | 10200 | 3.0711 | 1.0 |
3.3327 | 11.15 | 10300 | 3.0770 | 1.0 |
3.2911 | 11.26 | 10400 | 3.0325 | 1.0 |
3.2904 | 11.36 | 10500 | 2.9986 | 1.0 |
3.2709 | 11.47 | 10600 | 2.9960 | 1.0 |
3.2437 | 11.58 | 10700 | 2.9695 | 1.0 |
3.2532 | 11.69 | 10800 | 2.9565 | 1.0 |
3.2359 | 11.8 | 10900 | 2.9660 | 1.0 |
3.227 | 11.9 | 11000 | 2.9494 | 1.0 |
3.2292 | 12.01 | 11100 | 2.9384 | 1.0 |
3.197 | 12.12 | 11200 | 2.9342 | 1.0 |
3.183 | 12.23 | 11300 | 2.9108 | 1.0 |
3.1583 | 12.34 | 11400 | 2.8785 | 1.0 |
3.1501 | 12.45 | 11500 | 2.8748 | 1.0 |
3.1695 | 12.55 | 11600 | 2.8649 | 1.0 |
3.1341 | 12.66 | 11700 | 2.8779 | 1.0 |
3.141 | 12.77 | 11800 | 2.8420 | 1.0 |
3.113 | 12.88 | 11900 | 2.8088 | 1.0 |
3.1242 | 12.99 | 12000 | 2.7891 | 1.0 |
3.1234 | 13.1 | 12100 | 2.7859 | 1.0 |
3.1063 | 13.2 | 12200 | 2.7808 | 1.0 |
3.0785 | 13.31 | 12300 | 2.7735 | 1.0 |
3.0778 | 13.42 | 12400 | 2.7591 | 1.0 |
3.0559 | 13.53 | 12500 | 2.7519 | 1.0 |
3.046 | 13.64 | 12600 | 2.7228 | 1.0 |
3.0558 | 13.74 | 12700 | 2.7294 | 1.0 |
3.0489 | 13.85 | 12800 | 2.7090 | 1.0 |
3.0287 | 13.96 | 12900 | 2.7024 | 1.0 |
2.9927 | 14.07 | 13000 | 2.6963 | 1.0 |
2.9912 | 14.18 | 13100 | 2.6688 | 1.0 |
2.9816 | 14.29 | 13200 | 2.6834 | 1.0 |
2.966 | 14.39 | 13300 | 2.6762 | 1.0 |
2.9625 | 14.5 | 13400 | 2.6657 | 1.0 |
2.9827 | 14.61 | 13500 | 2.6598 | 1.0 |
2.9538 | 14.72 | 13600 | 2.6407 | 1.0 |
2.9524 | 14.83 | 13700 | 2.6399 | 1.0 |
2.9379 | 14.93 | 13800 | 2.6179 | 1.0 |
3.0388 | 15.04 | 13900 | 2.6130 | 1.0 |
2.9352 | 15.15 | 14000 | 2.6224 | 1.0 |
2.9172 | 15.26 | 14100 | 2.5905 | 1.0 |
2.9082 | 15.37 | 14200 | 2.5991 | 1.0 |
2.9566 | 15.48 | 14300 | 2.6069 | 1.0 |
2.9068 | 15.58 | 14400 | 2.5780 | 1.0 |
2.8904 | 15.69 | 14500 | 2.5782 | 1.0 |
2.8644 | 15.8 | 14600 | 2.5583 | 1.0 |
2.8932 | 15.91 | 14700 | 2.5593 | 1.0 |
2.8795 | 16.02 | 14800 | 2.5365 | 1.0 |
2.9069 | 16.13 | 14900 | 2.5330 | 1.0 |
2.9361 | 16.23 | 15000 | 2.5361 | 1.0 |
2.8348 | 16.34 | 15100 | 2.5445 | 1.0 |
2.851 | 16.45 | 15200 | 2.5289 | 1.0 |
2.864 | 16.56 | 15300 | 2.5193 | 1.0 |
2.8703 | 16.67 | 15400 | 2.5170 | 1.0 |
2.8326 | 16.77 | 15500 | 2.5213 | 1.0 |
2.8865 | 16.88 | 15600 | 2.5121 | 1.0 |
2.8495 | 16.99 | 15700 | 2.4892 | 1.0 |
2.8127 | 17.1 | 15800 | 2.4909 | 1.0 |
2.9142 | 17.21 | 15900 | 2.4761 | 1.0 |
2.7825 | 17.32 | 16000 | 2.4887 | 1.0 |
2.8134 | 17.42 | 16100 | 2.4658 | 1.0 |
2.826 | 17.53 | 16200 | 2.4658 | 1.0 |
2.812 | 17.64 | 16300 | 2.4666 | 1.0 |
2.7825 | 17.75 | 16400 | 2.4539 | 1.0 |
2.7964 | 17.86 | 16500 | 2.4550 | 1.0 |
2.8023 | 17.96 | 16600 | 2.4428 | 1.0 |
2.7691 | 18.07 | 16700 | 2.4448 | 1.0 |
2.7506 | 18.18 | 16800 | 2.4347 | 1.0 |
2.7784 | 18.29 | 16900 | 2.4214 | 1.0 |
2.755 | 18.4 | 17000 | 2.4309 | 1.0 |
2.7511 | 18.51 | 17100 | 2.4283 | 1.0 |
2.7425 | 18.61 | 17200 | 2.4294 | 1.0 |
2.7774 | 18.72 | 17300 | 2.4062 | 1.0 |
2.749 | 18.83 | 17400 | 2.4113 | 1.0 |
2.7407 | 18.94 | 17500 | 2.3999 | 1.0 |
2.7492 | 19.05 | 17600 | 2.4046 | 1.0 |
2.7538 | 19.16 | 17700 | 2.3945 | 1.0 |
2.7207 | 19.26 | 17800 | 2.3851 | 1.0 |
2.7176 | 19.37 | 17900 | 2.3954 | 1.0 |
2.7333 | 19.48 | 18000 | 2.3855 | 1.0 |
2.7192 | 19.59 | 18100 | 2.3802 | 1.0 |
2.7252 | 19.7 | 18200 | 2.3535 | 1.0 |
2.7002 | 19.8 | 18300 | 2.3808 | 1.0 |
2.6591 | 19.91 | 18400 | 2.3590 | 1.0 |
2.7684 | 20.02 | 18500 | 2.3627 | 1.0 |
2.6802 | 20.13 | 18600 | 2.3468 | 1.0 |
2.6649 | 20.24 | 18700 | 2.3405 | 1.0 |
2.6886 | 20.35 | 18800 | 2.3358 | 1.0 |
2.7023 | 20.45 | 18900 | 2.3514 | 1.0 |
2.6993 | 20.56 | 19000 | 2.3433 | 1.0 |
2.691 | 20.67 | 19100 | 2.3498 | 1.0 |
2.6666 | 20.78 | 19200 | 2.3457 | 1.0 |
2.6829 | 20.89 | 19300 | 2.3451 | 1.0 |
2.7203 | 21.0 | 19400 | 2.3287 | 1.0 |
2.6738 | 21.1 | 19500 | 2.3205 | 1.0 |
2.6781 | 21.21 | 19600 | 2.3264 | 1.0 |
2.7018 | 21.32 | 19700 | 2.3217 | 1.0 |
2.6642 | 21.43 | 19800 | 2.3114 | 1.0 |
2.6662 | 21.54 | 19900 | 2.3188 | 1.0 |
2.6636 | 21.64 | 20000 | 2.3180 | 1.0 |
2.6553 | 21.75 | 20100 | 2.3095 | 1.0 |
2.6369 | 21.86 | 20200 | 2.3066 | 1.0 |
2.6355 | 21.97 | 20300 | 2.3048 | 1.0 |
2.6317 | 22.08 | 20400 | 2.3080 | 1.0 |
2.6631 | 22.19 | 20500 | 2.2931 | 1.0 |
2.6469 | 22.29 | 20600 | 2.2910 | 1.0 |
2.6401 | 22.4 | 20700 | 2.2857 | 1.0 |
2.6434 | 22.51 | 20800 | 2.2951 | 1.0 |
2.635 | 22.62 | 20900 | 2.2924 | 1.0 |
2.637 | 22.73 | 21000 | 2.2831 | 1.0 |
2.6249 | 22.84 | 21100 | 2.2897 | 1.0 |
2.6293 | 22.94 | 21200 | 2.2790 | 1.0 |
2.6482 | 23.05 | 21300 | 2.2821 | 1.0 |
2.6204 | 23.16 | 21400 | 2.2709 | 1.0 |
2.6337 | 23.27 | 21500 | 2.2675 | 1.0 |
2.6339 | 23.38 | 21600 | 2.2658 | 1.0 |
2.6169 | 23.48 | 21700 | 2.2701 | 1.0 |
2.6038 | 23.59 | 21800 | 2.2774 | 1.0 |
2.6255 | 23.7 | 21900 | 2.2740 | 1.0 |
2.6029 | 23.81 | 22000 | 2.2777 | 1.0 |
2.6045 | 23.92 | 22100 | 2.2663 | 1.0 |
2.6367 | 24.03 | 22200 | 2.2627 | 1.0 |
2.6071 | 24.13 | 22300 | 2.2574 | 1.0 |
2.6057 | 24.24 | 22400 | 2.2477 | 1.0 |
2.6167 | 24.35 | 22500 | 2.2592 | 1.0 |
2.607 | 24.46 | 22600 | 2.2514 | 1.0 |
2.5864 | 24.57 | 22700 | 2.2514 | 1.0 |
2.6053 | 24.67 | 22800 | 2.2475 | 1.0 |
2.616 | 24.78 | 22900 | 2.2436 | 1.0 |
2.5876 | 24.89 | 23000 | 2.2511 | 1.0 |
2.5977 | 25.0 | 23100 | 2.2461 | 1.0 |
2.6238 | 25.11 | 23200 | 2.2404 | 1.0 |
2.566 | 25.22 | 23300 | 2.2471 | 1.0 |
2.5851 | 25.32 | 23400 | 2.2444 | 1.0 |
2.5916 | 25.43 | 23500 | 2.2402 | 1.0 |
2.6528 | 25.54 | 23600 | 2.2418 | 1.0 |
2.5831 | 25.65 | 23700 | 2.2314 | 1.0 |
2.5725 | 25.76 | 23800 | 2.2433 | 1.0 |
2.5842 | 25.87 | 23900 | 2.2260 | 1.0 |
2.604 | 25.97 | 24000 | 2.2392 | 1.0 |
2.5801 | 26.08 | 24100 | 2.2339 | 1.0 |
2.5798 | 26.19 | 24200 | 2.2354 | 1.0 |
2.5747 | 26.3 | 24300 | 2.2305 | 1.0 |
2.5879 | 26.41 | 24400 | 2.2272 | 1.0 |
2.5494 | 26.51 | 24500 | 2.2319 | 1.0 |
2.5789 | 26.62 | 24600 | 2.2228 | 1.0 |
2.573 | 26.73 | 24700 | 2.2305 | 1.0 |
2.5864 | 26.84 | 24800 | 2.2254 | 1.0 |
2.5658 | 26.95 | 24900 | 2.2154 | 1.0 |
2.5766 | 27.06 | 25000 | 2.2209 | 1.0 |
2.5468 | 27.16 | 25100 | 2.2197 | 1.0 |
2.5867 | 27.27 | 25200 | 2.2148 | 1.0 |
2.5573 | 27.38 | 25300 | 2.2282 | 1.0 |
2.5742 | 27.49 | 25400 | 2.2245 | 1.0 |
2.5537 | 27.6 | 25500 | 2.2233 | 1.0 |
2.5518 | 27.71 | 25600 | 2.2207 | 1.0 |
2.5823 | 27.81 | 25700 | 2.2125 | 1.0 |
2.5611 | 27.92 | 25800 | 2.2198 | 1.0 |
2.5933 | 28.03 | 25900 | 2.2153 | 1.0 |
2.5271 | 28.14 | 26000 | 2.2138 | 1.0 |
2.5768 | 28.25 | 26100 | 2.2167 | 1.0 |
2.5649 | 28.35 | 26200 | 2.2108 | 1.0 |
2.5522 | 28.46 | 26300 | 2.2150 | 1.0 |
2.5723 | 28.57 | 26400 | 2.2162 | 1.0 |
2.5799 | 28.68 | 26500 | 2.2145 | 1.0 |
2.5673 | 28.79 | 26600 | 2.2153 | 1.0 |
2.5584 | 28.9 | 26700 | 2.2171 | 1.0 |
2.5547 | 29.0 | 26800 | 2.2100 | 1.0 |
2.5643 | 29.11 | 26900 | 2.2104 | 1.0 |
2.6011 | 29.22 | 27000 | 2.2113 | 1.0 |
2.5506 | 29.33 | 27100 | 2.2171 | 1.0 |
2.5858 | 29.44 | 27200 | 2.2129 | 1.0 |
2.5437 | 29.55 | 27300 | 2.2138 | 1.0 |
2.5627 | 29.65 | 27400 | 2.2167 | 1.0 |
2.5552 | 29.76 | 27500 | 2.2144 | 1.0 |
2.5578 | 29.87 | 27600 | 2.2145 | 1.0 |
2.5628 | 29.98 | 27700 | 2.2139 | 1.0 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.12.0
- Datasets 2.7.1
- Tokenizers 0.13.2
- Downloads last month
- 7
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.