|
--- |
|
library_name: transformers |
|
language: |
|
- zul |
|
license: cc-by-nc-4.0 |
|
base_model: facebook/mms-1b-all |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- NCHLT_speech_corpus |
|
metrics: |
|
- wer |
|
model-index: |
|
- name: facebook mms-1b-all zulu - Beijuka Bruno |
|
results: |
|
- task: |
|
name: Automatic Speech Recognition |
|
type: automatic-speech-recognition |
|
dataset: |
|
name: NCHLT_speech_corpus/Zulu |
|
type: NCHLT_speech_corpus |
|
metrics: |
|
- name: Wer |
|
type: wer |
|
value: 0.3739586979348967 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# facebook mms-1b-all zulu - Beijuka Bruno |
|
|
|
This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the NCHLT_speech_corpus/Zulu dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.2337 |
|
- Model Preparation Time: 0.0185 |
|
- Wer: 0.3740 |
|
- Cer: 0.0682 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0003 |
|
- train_batch_size: 4 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 8 |
|
- total_train_batch_size: 32 |
|
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 100 |
|
- num_epochs: 100 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Model Preparation Time | Wer | Cer | |
|
|:-------------:|:-------:|:----:|:---------------:|:----------------------:|:------:|:------:| |
|
| 134.9856 | 0.9796 | 30 | 13.9196 | 0.0185 | 1.0 | 1.1928 | |
|
| 74.6875 | 1.9796 | 60 | 7.4763 | 0.0185 | 1.0 | 0.9660 | |
|
| 48.924 | 2.9796 | 90 | 5.0082 | 0.0185 | 1.0 | 0.9261 | |
|
| 34.5938 | 3.9796 | 120 | 3.8247 | 0.0185 | 1.0 | 0.9220 | |
|
| 28.0008 | 4.9796 | 150 | 3.2622 | 0.0185 | 1.0 | 0.9262 | |
|
| 25.6264 | 5.9796 | 180 | 3.0235 | 0.0185 | 0.9996 | 0.9055 | |
|
| 23.2532 | 6.9796 | 210 | 2.7683 | 0.0185 | 0.9797 | 0.7992 | |
|
| 12.3233 | 7.9796 | 240 | 0.2848 | 0.0185 | 0.3328 | 0.0541 | |
|
| 2.5333 | 8.9796 | 270 | 0.1889 | 0.0185 | 0.2788 | 0.0421 | |
|
| 2.1275 | 9.9796 | 300 | 0.1637 | 0.0185 | 0.2469 | 0.0373 | |
|
| 1.9296 | 10.9796 | 330 | 0.1574 | 0.0185 | 0.2447 | 0.0368 | |
|
| 1.7347 | 11.9796 | 360 | 0.1516 | 0.0185 | 0.2398 | 0.0364 | |
|
| 1.7104 | 12.9796 | 390 | 0.1484 | 0.0185 | 0.2364 | 0.0355 | |
|
| 1.6018 | 13.9796 | 420 | 0.1426 | 0.0185 | 0.2300 | 0.0344 | |
|
| 1.5881 | 14.9796 | 450 | 0.1413 | 0.0185 | 0.2285 | 0.0337 | |
|
| 1.5097 | 15.9796 | 480 | 0.1381 | 0.0185 | 0.2289 | 0.0337 | |
|
| 1.4149 | 16.9796 | 510 | 0.1313 | 0.0185 | 0.2180 | 0.0321 | |
|
| 1.4524 | 17.9796 | 540 | 0.1327 | 0.0185 | 0.2128 | 0.0313 | |
|
| 1.3885 | 18.9796 | 570 | 0.1311 | 0.0185 | 0.2146 | 0.0323 | |
|
| 1.3654 | 19.9796 | 600 | 0.1297 | 0.0185 | 0.2101 | 0.0312 | |
|
| 1.2783 | 20.9796 | 630 | 0.1337 | 0.0185 | 0.2135 | 0.0313 | |
|
| 1.2367 | 21.9796 | 660 | 0.1309 | 0.0185 | 0.2086 | 0.0310 | |
|
| 1.219 | 22.9796 | 690 | 0.1282 | 0.0185 | 0.2124 | 0.0311 | |
|
| 1.1463 | 23.9796 | 720 | 0.1277 | 0.0185 | 0.2075 | 0.0307 | |
|
| 1.1602 | 24.9796 | 750 | 0.1281 | 0.0185 | 0.2101 | 0.0310 | |
|
| 1.1487 | 25.9796 | 780 | 0.1256 | 0.0185 | 0.2019 | 0.0297 | |
|
| 1.1645 | 26.9796 | 810 | 0.1259 | 0.0185 | 0.1985 | 0.0296 | |
|
| 1.0974 | 27.9796 | 840 | 0.1268 | 0.0185 | 0.2023 | 0.0294 | |
|
| 1.0918 | 28.9796 | 870 | 0.1356 | 0.0185 | 0.2079 | 0.0311 | |
|
| 1.0377 | 29.9796 | 900 | 0.1270 | 0.0185 | 0.2 | 0.0300 | |
|
| 0.9772 | 30.9796 | 930 | 0.1260 | 0.0185 | 0.1944 | 0.0289 | |
|
| 0.9718 | 31.9796 | 960 | 0.1258 | 0.0185 | 0.1940 | 0.0289 | |
|
| 0.9485 | 32.9796 | 990 | 0.1258 | 0.0185 | 0.1962 | 0.0289 | |
|
| 0.9636 | 33.9796 | 1020 | 0.1256 | 0.0185 | 0.1910 | 0.0283 | |
|
| 0.9122 | 34.9796 | 1050 | 0.1277 | 0.0185 | 0.1932 | 0.0289 | |
|
| 0.9422 | 35.9796 | 1080 | 0.1263 | 0.0185 | 0.1906 | 0.0285 | |
|
| 0.9554 | 36.9796 | 1110 | 0.1326 | 0.0185 | 0.1940 | 0.0292 | |
|
| 0.9435 | 37.9796 | 1140 | 0.1301 | 0.0185 | 0.1925 | 0.0287 | |
|
| 0.8582 | 38.9796 | 1170 | 0.1279 | 0.0185 | 0.1895 | 0.0281 | |
|
| 0.8792 | 39.9796 | 1200 | 0.1321 | 0.0185 | 0.1944 | 0.0299 | |
|
| 0.8537 | 40.9796 | 1230 | 0.1294 | 0.0185 | 0.1932 | 0.0287 | |
|
| 0.8037 | 41.9796 | 1260 | 0.1324 | 0.0185 | 0.1936 | 0.0289 | |
|
| 0.7627 | 42.9796 | 1290 | 0.1347 | 0.0185 | 0.1992 | 0.0295 | |
|
| 0.836 | 43.9796 | 1320 | 0.1356 | 0.0185 | 0.1996 | 0.0292 | |
|
| 0.8203 | 44.9796 | 1350 | 0.1348 | 0.0185 | 0.1951 | 0.0282 | |
|
| 0.8289 | 45.9796 | 1380 | 0.1340 | 0.0185 | 0.1974 | 0.0289 | |
|
| 0.794 | 46.9796 | 1410 | 0.1305 | 0.0185 | 0.1902 | 0.0280 | |
|
| 0.7716 | 47.9796 | 1440 | 0.1307 | 0.0185 | 0.1902 | 0.0276 | |
|
| 0.7461 | 48.9796 | 1470 | 0.1327 | 0.0185 | 0.1925 | 0.0279 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.47.1 |
|
- Pytorch 2.1.0+cu118 |
|
- Datasets 3.2.0 |
|
- Tokenizers 0.21.0 |
|
|
