---
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-300m
tags:
- generated_from_trainer
datasets:
- common_voice_17_0
metrics:
- wer
model-index:
- name: xls-r-300-cv17-polish-adap-cs
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: common_voice_17_0
      type: common_voice_17_0
      config: pl
      split: validation
      args: pl
    metrics:
    - name: Wer
      type: wer
      value: 0.3181674482322567
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/badr-nlp/xlsr-continual-finetuning-polish/runs/gugvjjo9)
# xls-r-300-cv17-polish-adap-cs

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice_17_0 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4585
- Wer: 0.3182
- Cer: 0.0713

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|
| 3.5986        | 1.6   | 100  | 3.9654          | 0.9986 | 0.9660 |
| 3.2886        | 3.2   | 200  | 3.4889          | 1.0    | 1.0    |
| 3.1683        | 4.8   | 300  | 3.1937          | 0.9946 | 0.9735 |
| 2.7362        | 6.4   | 400  | 2.6853          | 1.0    | 0.8424 |
| 0.6269        | 8.0   | 500  | 0.5183          | 0.5745 | 0.1381 |
| 0.2661        | 9.6   | 600  | 0.4218          | 0.4551 | 0.1048 |
| 0.1646        | 11.2  | 700  | 0.4160          | 0.4211 | 0.0985 |
| 0.1197        | 12.8  | 800  | 0.4793          | 0.4578 | 0.1072 |
| 0.1925        | 14.4  | 900  | 0.4402          | 0.4283 | 0.0969 |
| 0.1132        | 16.0  | 1000 | 0.4253          | 0.3909 | 0.0906 |
| 0.0851        | 17.6  | 1100 | 0.4609          | 0.3951 | 0.0921 |
| 0.0799        | 19.2  | 1200 | 0.4453          | 0.3944 | 0.0907 |
| 0.0657        | 20.8  | 1300 | 0.4681          | 0.3846 | 0.0887 |
| 0.1188        | 22.4  | 1400 | 0.4575          | 0.3785 | 0.0873 |
| 0.1088        | 24.0  | 1500 | 0.4649          | 0.3824 | 0.0882 |
| 0.0698        | 25.6  | 1600 | 0.4496          | 0.3611 | 0.0817 |
| 0.0575        | 27.2  | 1700 | 0.4459          | 0.3585 | 0.0822 |
| 0.0705        | 28.8  | 1800 | 0.4542          | 0.3608 | 0.0820 |
| 0.0524        | 30.4  | 1900 | 0.4785          | 0.3549 | 0.0814 |
| 0.0338        | 32.0  | 2000 | 0.4566          | 0.3521 | 0.0801 |
| 0.0357        | 33.6  | 2100 | 0.4597          | 0.3472 | 0.0783 |
| 0.0477        | 35.2  | 2200 | 0.4626          | 0.3451 | 0.0788 |
| 0.0478        | 36.8  | 2300 | 0.4730          | 0.3375 | 0.0765 |
| 0.0568        | 38.4  | 2400 | 0.4713          | 0.3333 | 0.0749 |
| 0.0217        | 40.0  | 2500 | 0.4701          | 0.3324 | 0.0755 |
| 0.0404        | 41.6  | 2600 | 0.4585          | 0.3278 | 0.0740 |
| 0.0118        | 43.2  | 2700 | 0.4656          | 0.3259 | 0.0736 |
| 0.0374        | 44.8  | 2800 | 0.4625          | 0.3249 | 0.0731 |
| 0.0417        | 46.4  | 2900 | 0.4599          | 0.3206 | 0.0721 |
| 0.0378        | 48.0  | 3000 | 0.4614          | 0.3195 | 0.0717 |
| 0.0381        | 49.6  | 3100 | 0.4585          | 0.3182 | 0.0713 |


### Framework versions

- Transformers 4.42.0.dev0
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1