metadata
license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v2-base-pretrained_lr5e-5_at0.8_da0.4
results: []
w2v2-base-pretrained_lr5e-5_at0.8_da0.4
This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 5.6336
- Wer: 0.8377
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 4000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 27.1555 | 13.16 | 250 | 16.9359 | 1.0 |
| 8.9276 | 26.32 | 500 | 9.0668 | 1.0 |
| 6.6067 | 39.47 | 750 | 8.0040 | 1.0 |
| 3.8775 | 52.63 | 1000 | 5.5239 | 1.0 |
| 3.1858 | 65.79 | 1250 | 5.5803 | 1.0 |
| 3.0238 | 78.95 | 1500 | 5.5821 | 1.0004 |
| 2.7704 | 92.11 | 1750 | 5.6283 | 1.0068 |
| 2.4867 | 105.26 | 2000 | 5.4805 | 0.9855 |
| 2.2251 | 118.42 | 2250 | 5.6338 | 0.9645 |
| 1.9731 | 131.58 | 2500 | 5.5967 | 0.9197 |
| 1.752 | 144.74 | 2750 | 5.5503 | 0.8949 |
| 1.5403 | 157.89 | 3000 | 5.4723 | 0.8663 |
| 1.3664 | 171.05 | 3250 | 5.5707 | 0.8603 |
| 1.2337 | 184.21 | 3500 | 5.5728 | 0.8449 |
| 1.1468 | 197.37 | 3750 | 5.6135 | 0.8402 |
| 1.0931 | 210.53 | 4000 | 5.6336 | 0.8377 |
Framework versions
- Transformers 4.35.0
- Pytorch 2.0.0
- Datasets 2.14.6
- Tokenizers 0.14.1