---
license: cc-by-nc-4.0
tags:
- generated_from_trainer
base_model: facebook/mms-1b-all
metrics:
- wer
model-index:
- name: wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching

This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3113
- Wer: 0.4087
- Cer: 0.0910

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- training_steps: 10000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Wer    | Cer    |
|:-------------:|:-------:|:-----:|:---------------:|:------:|:------:|
| 1.4219        | 4.5351  | 500   | 0.4570          | 0.5677 | 0.1335 |
| 0.5951        | 9.0703  | 1000  | 0.4008          | 0.5142 | 0.1186 |
| 0.5314        | 13.6054 | 1500  | 0.3725          | 0.4942 | 0.1126 |
| 0.4916        | 18.1406 | 2000  | 0.3626          | 0.4692 | 0.1067 |
| 0.4563        | 22.6757 | 2500  | 0.3465          | 0.4540 | 0.1035 |
| 0.4331        | 27.2109 | 3000  | 0.3310          | 0.4455 | 0.1010 |
| 0.4129        | 31.7460 | 3500  | 0.3283          | 0.4516 | 0.1019 |
| 0.394         | 36.2812 | 4000  | 0.3289          | 0.4482 | 0.0994 |
| 0.3715        | 40.8163 | 4500  | 0.3203          | 0.4374 | 0.0985 |
| 0.3646        | 45.3515 | 5000  | 0.3109          | 0.4327 | 0.0966 |
| 0.3508        | 49.8866 | 5500  | 0.3136          | 0.4276 | 0.0958 |
| 0.3376        | 54.4218 | 6000  | 0.3198          | 0.4246 | 0.0950 |
| 0.3283        | 58.9569 | 6500  | 0.3203          | 0.4232 | 0.0943 |
| 0.3222        | 63.4921 | 7000  | 0.3126          | 0.4134 | 0.0932 |
| 0.3104        | 68.0272 | 7500  | 0.3140          | 0.4168 | 0.0933 |
| 0.3026        | 72.5624 | 8000  | 0.3136          | 0.4110 | 0.0920 |
| 0.3003        | 77.0975 | 8500  | 0.3137          | 0.4175 | 0.0926 |
| 0.2896        | 81.6327 | 9000  | 0.3150          | 0.4107 | 0.0912 |
| 0.2885        | 86.1678 | 9500  | 0.3110          | 0.4090 | 0.0914 |
| 0.2869        | 90.7029 | 10000 | 0.3113          | 0.4087 | 0.0910 |


### Framework versions

- Transformers 4.41.0.dev0
- Pytorch 2.2.2
- Datasets 2.19.0
- Tokenizers 0.19.1