wav2vec2-large-mms-1b-russian-colab
This model is a fine-tuned version of facebook/mms-1b-all on the common_voice_6_1 dataset. It achieves the following results on the evaluation set:
- Loss: 0.1305
- Wer: 0.1798
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 4
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
3.4789 | 0.21 | 100 | 0.1676 | 0.2126 |
0.2042 | 0.41 | 200 | 0.1507 | 0.2010 |
0.1991 | 0.62 | 300 | 0.1460 | 0.1989 |
0.1882 | 0.83 | 400 | 0.1464 | 0.1992 |
0.1804 | 1.03 | 500 | 0.1402 | 0.1900 |
0.1809 | 1.24 | 600 | 0.1404 | 0.1890 |
0.1787 | 1.45 | 700 | 0.1392 | 0.1873 |
0.1781 | 1.65 | 800 | 0.1413 | 0.1926 |
0.1767 | 1.86 | 900 | 0.1411 | 0.1883 |
0.1779 | 2.07 | 1000 | 0.1375 | 0.1853 |
0.1749 | 2.27 | 1100 | 0.1356 | 0.1845 |
0.1749 | 2.48 | 1200 | 0.1343 | 0.1836 |
0.1652 | 2.69 | 1300 | 0.1339 | 0.1826 |
0.1677 | 2.89 | 1400 | 0.1334 | 0.1826 |
0.1764 | 3.1 | 1500 | 0.1333 | 0.1832 |
0.1697 | 3.31 | 1600 | 0.1320 | 0.1811 |
0.1678 | 3.51 | 1700 | 0.1313 | 0.1804 |
0.165 | 3.72 | 1800 | 0.1308 | 0.1799 |
0.1641 | 3.93 | 1900 | 0.1305 | 0.1798 |
Framework versions
- Transformers 4.39.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 9