metadata
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xls-r-300m-or
results: []
wav2vec2-large-xls-r-300m-or
This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice dataset. It achieves the following results on the evaluation set:
- Loss: 1.6618
- Wer: 0.5166
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.12
- num_epochs: 240
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 6.0493 | 23.53 | 400 | 2.9728 | 1.0 |
| 0.5306 | 47.06 | 800 | 1.2895 | 0.6138 |
| 0.1253 | 70.59 | 1200 | 1.6854 | 0.5703 |
| 0.0763 | 94.12 | 1600 | 1.9433 | 0.5870 |
| 0.0552 | 117.65 | 2000 | 1.4393 | 0.5575 |
| 0.0382 | 141.18 | 2400 | 1.4665 | 0.5537 |
| 0.0286 | 164.71 | 2800 | 1.5441 | 0.5320 |
| 0.0212 | 188.24 | 3200 | 1.6502 | 0.5115 |
| 0.0168 | 211.76 | 3600 | 1.6411 | 0.5332 |
| 0.0129 | 235.29 | 4000 | 1.6618 | 0.5166 |
Framework versions
- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3