|
--- |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- common_voice |
|
model-index: |
|
- name: '' |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# |
|
|
|
This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.7564 |
|
- Wer: 0.7201 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 7.5e-05 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 2 |
|
- total_train_batch_size: 16 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 1000 |
|
- num_epochs: 500.0 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer | |
|
|:-------------:|:------:|:-----:|:---------------:|:------:| |
|
| 18.5133 | 4.17 | 100 | 12.0376 | 1.0 | |
|
| 7.9152 | 8.33 | 200 | 6.2406 | 1.0 | |
|
| 5.363 | 12.5 | 300 | 4.6649 | 1.0 | |
|
| 4.05 | 16.67 | 400 | 3.5798 | 1.0 | |
|
| 3.3467 | 20.83 | 500 | 3.1930 | 1.0 | |
|
| 3.1638 | 25.0 | 600 | 3.0984 | 1.0 | |
|
| 3.043 | 29.17 | 700 | 2.9707 | 0.9983 | |
|
| 2.9566 | 33.33 | 800 | 2.9296 | 1.0 | |
|
| 2.8994 | 37.5 | 900 | 2.8290 | 0.9983 | |
|
| 2.6469 | 41.67 | 1000 | 2.2443 | 0.9765 | |
|
| 2.1557 | 45.83 | 1100 | 1.5079 | 0.8204 | |
|
| 1.7524 | 50.0 | 1200 | 1.2472 | 0.7594 | |
|
| 1.54 | 54.17 | 1300 | 1.1612 | 0.7097 | |
|
| 1.3985 | 58.33 | 1400 | 1.1158 | 0.7358 | |
|
| 1.2869 | 62.5 | 1500 | 1.0454 | 0.6931 | |
|
| 1.1952 | 66.67 | 1600 | 1.0130 | 0.6853 | |
|
| 1.1022 | 70.83 | 1700 | 1.0176 | 0.6966 | |
|
| 1.0346 | 75.0 | 1800 | 1.0053 | 0.6818 | |
|
| 0.9707 | 79.17 | 1900 | 1.0224 | 0.6713 | |
|
| 0.917 | 83.33 | 2000 | 1.0383 | 0.6530 | |
|
| 0.8574 | 87.5 | 2100 | 1.0632 | 0.6757 | |
|
| 0.8021 | 91.67 | 2200 | 1.0126 | 0.6443 | |
|
| 0.7563 | 95.83 | 2300 | 1.0678 | 0.6713 | |
|
| 0.709 | 100.0 | 2400 | 1.0756 | 0.6757 | |
|
| 0.6775 | 104.17 | 2500 | 1.0397 | 0.6914 | |
|
| 0.6325 | 108.33 | 2600 | 1.0556 | 0.6513 | |
|
| 0.617 | 112.5 | 2700 | 1.1219 | 0.6922 | |
|
| 0.5801 | 116.67 | 2800 | 1.1125 | 0.6870 | |
|
| 0.5367 | 120.83 | 2900 | 1.1397 | 0.6565 | |
|
| 0.5132 | 125.0 | 3000 | 1.1678 | 0.7001 | |
|
| 0.4948 | 129.17 | 3100 | 1.1276 | 0.6643 | |
|
| 0.457 | 133.33 | 3200 | 1.1465 | 0.6792 | |
|
| 0.4538 | 137.5 | 3300 | 1.1435 | 0.6809 | |
|
| 0.4227 | 141.67 | 3400 | 1.1457 | 0.6600 | |
|
| 0.4083 | 145.83 | 3500 | 1.1793 | 0.6652 | |
|
| 0.3965 | 150.0 | 3600 | 1.2435 | 0.6879 | |
|
| 0.382 | 154.17 | 3700 | 1.2229 | 0.6827 | |
|
| 0.3452 | 158.33 | 3800 | 1.2297 | 0.6879 | |
|
| 0.3434 | 162.5 | 3900 | 1.2350 | 0.6888 | |
|
| 0.3276 | 166.67 | 4000 | 1.2186 | 0.6922 | |
|
| 0.3052 | 170.83 | 4100 | 1.2338 | 0.6870 | |
|
| 0.3025 | 175.0 | 4200 | 1.2724 | 0.7079 | |
|
| 0.2916 | 179.17 | 4300 | 1.2758 | 0.6975 | |
|
| 0.2709 | 183.33 | 4400 | 1.2726 | 0.6748 | |
|
| 0.2707 | 187.5 | 4500 | 1.2643 | 0.6957 | |
|
| 0.262 | 191.67 | 4600 | 1.3214 | 0.7132 | |
|
| 0.2453 | 195.83 | 4700 | 1.2953 | 0.6861 | |
|
| 0.248 | 200.0 | 4800 | 1.3622 | 0.6774 | |
|
| 0.2325 | 204.17 | 4900 | 1.3594 | 0.6835 | |
|
| 0.2124 | 208.33 | 5000 | 1.3367 | 0.6652 | |
|
| 0.2253 | 212.5 | 5100 | 1.4157 | 0.6879 | |
|
| 0.2059 | 216.67 | 5200 | 1.4360 | 0.7132 | |
|
| 0.1951 | 220.83 | 5300 | 1.4606 | 0.7158 | |
|
| 0.1861 | 225.0 | 5400 | 1.4695 | 0.7018 | |
|
| 0.1916 | 229.17 | 5500 | 1.4031 | 0.6739 | |
|
| 0.1822 | 233.33 | 5600 | 1.4426 | 0.6870 | |
|
| 0.1684 | 237.5 | 5700 | 1.4069 | 0.7053 | |
|
| 0.1719 | 241.67 | 5800 | 1.4766 | 0.6966 | |
|
| 0.1569 | 245.83 | 5900 | 1.4509 | 0.6931 | |
|
| 0.159 | 250.0 | 6000 | 1.4467 | 0.7097 | |
|
| 0.1476 | 254.17 | 6100 | 1.4617 | 0.6870 | |
|
| 0.1497 | 258.33 | 6200 | 1.4460 | 0.6844 | |
|
| 0.1446 | 262.5 | 6300 | 1.5557 | 0.7088 | |
|
| 0.1389 | 266.67 | 6400 | 1.4886 | 0.7140 | |
|
| 0.1331 | 270.83 | 6500 | 1.5526 | 0.7062 | |
|
| 0.1344 | 275.0 | 6600 | 1.5419 | 0.7027 | |
|
| 0.1198 | 279.17 | 6700 | 1.5641 | 0.7001 | |
|
| 0.1242 | 283.33 | 6800 | 1.5390 | 0.7062 | |
|
| 0.12 | 287.5 | 6900 | 1.5406 | 0.7105 | |
|
| 0.1096 | 291.67 | 7000 | 1.5737 | 0.6975 | |
|
| 0.113 | 295.83 | 7100 | 1.5495 | 0.7210 | |
|
| 0.108 | 300.0 | 7200 | 1.5375 | 0.6949 | |
|
| 0.1072 | 304.17 | 7300 | 1.5337 | 0.7010 | |
|
| 0.0979 | 308.33 | 7400 | 1.5927 | 0.7062 | |
|
| 0.0983 | 312.5 | 7500 | 1.5882 | 0.6844 | |
|
| 0.0977 | 316.67 | 7600 | 1.6189 | 0.6957 | |
|
| 0.0947 | 320.83 | 7700 | 1.5098 | 0.6818 | |
|
| 0.0996 | 325.0 | 7800 | 1.6269 | 0.7254 | |
|
| 0.0846 | 329.17 | 7900 | 1.6367 | 0.7088 | |
|
| 0.0953 | 333.33 | 8000 | 1.5965 | 0.7123 | |
|
| 0.0906 | 337.5 | 8100 | 1.6096 | 0.7123 | |
|
| 0.093 | 341.67 | 8200 | 1.5953 | 0.6983 | |
|
| 0.0784 | 345.83 | 8300 | 1.5884 | 0.6914 | |
|
| 0.0769 | 350.0 | 8400 | 1.5794 | 0.6870 | |
|
| 0.0782 | 354.17 | 8500 | 1.6581 | 0.6818 | |
|
| 0.0764 | 358.33 | 8600 | 1.6555 | 0.7088 | |
|
| 0.073 | 362.5 | 8700 | 1.6466 | 0.6931 | |
|
| 0.0703 | 366.67 | 8800 | 1.6615 | 0.7114 | |
|
| 0.0707 | 370.83 | 8900 | 1.6743 | 0.7079 | |
|
| 0.0647 | 375.0 | 9000 | 1.6452 | 0.7167 | |
|
| 0.0614 | 379.17 | 9100 | 1.7082 | 0.7123 | |
|
| 0.0646 | 383.33 | 9200 | 1.6848 | 0.7184 | |
|
| 0.0648 | 387.5 | 9300 | 1.6581 | 0.7088 | |
|
| 0.0625 | 391.67 | 9400 | 1.7315 | 0.7341 | |
|
| 0.0637 | 395.83 | 9500 | 1.6831 | 0.7027 | |
|
| 0.0558 | 400.0 | 9600 | 1.7159 | 0.7280 | |
|
| 0.0563 | 404.17 | 9700 | 1.7475 | 0.7158 | |
|
| 0.0568 | 408.33 | 9800 | 1.6776 | 0.6992 | |
|
| 0.0574 | 412.5 | 9900 | 1.7150 | 0.6983 | |
|
| 0.0561 | 416.67 | 10000 | 1.7315 | 0.7140 | |
|
| 0.0494 | 420.83 | 10100 | 1.6869 | 0.7219 | |
|
| 0.0495 | 425.0 | 10200 | 1.7500 | 0.7262 | |
|
| 0.0542 | 429.17 | 10300 | 1.7298 | 0.7271 | |
|
| 0.0509 | 433.33 | 10400 | 1.7334 | 0.7262 | |
|
| 0.046 | 437.5 | 10500 | 1.7048 | 0.7193 | |
|
| 0.0423 | 441.67 | 10600 | 1.7168 | 0.7193 | |
|
| 0.0477 | 445.83 | 10700 | 1.7388 | 0.7210 | |
|
| 0.0436 | 450.0 | 10800 | 1.7279 | 0.7167 | |
|
| 0.0466 | 454.17 | 10900 | 1.6968 | 0.7053 | |
|
| 0.0424 | 458.33 | 11000 | 1.7237 | 0.7184 | |
|
| 0.0447 | 462.5 | 11100 | 1.7218 | 0.7184 | |
|
| 0.0455 | 466.67 | 11200 | 1.7506 | 0.7219 | |
|
| 0.0446 | 470.83 | 11300 | 1.7542 | 0.7280 | |
|
| 0.043 | 475.0 | 11400 | 1.7501 | 0.7201 | |
|
| 0.0397 | 479.17 | 11500 | 1.7837 | 0.7245 | |
|
| 0.0402 | 483.33 | 11600 | 1.7762 | 0.7175 | |
|
| 0.039 | 487.5 | 11700 | 1.7771 | 0.7262 | |
|
| 0.0402 | 491.67 | 11800 | 1.7564 | 0.7219 | |
|
| 0.0368 | 495.83 | 11900 | 1.7553 | 0.7193 | |
|
| 0.0395 | 500.0 | 12000 | 1.7564 | 0.7201 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.17.0.dev0 |
|
- Pytorch 1.10.2+cu102 |
|
- Datasets 1.18.3 |
|
- Tokenizers 0.11.0 |
|
|
